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Foreword 


In  2009,  Air  Force  Force  Management  Division  (AF/AIPF)  asked  the  RAND  Corporation 
for  assistance  in  identifying  whether  the  existing  personnel  research  efforts  were  meeting  the 
broader  Air  Force’s  personnel  research  needs.  In  response,  RAND  Project  AIR  FORCE  con¬ 
ducted  a  series  of  interviews  with  key  personnel  in  the  Air  Force  to  identify  the  organizations 
involved  in  personnel-related  research  efforts.  Informed  hy  the  hndings,  the  RAND  team 
suggested  several  ways  to  improve  the  organizational  structure  to  reduce  duplication  and  help 
streamline  the  work. 

In  2011,  the  RAND  team  reported  its  recommendations  for  realignment  to  Air  Force 
leadership  in  a  draft  report  summarizing  the  state  of  the  relevant  organizations  as  of  2010.  This 
report  publishes  those  findings. 

Although  the  organizations  and  their  structures  may  have  changed  since  the  report  was 
produced  in  early  2012  (to  include  merging  of  the  Air  Force  Manpower  Agency  and  Air  Force 
Personnel  Center),  many  of  the  hndings  and  core  recommendations  of  this  work  still  stand. 

For  example,  the  report  emphasizes  the  need  for  the  following: 

•  increased  communication,  information  sharing,  and  data  sharing 

•  having  a  single  authority  with  clear  oversight  responsibility  for  all  the  personnel  research 

efforts  ongoing  in  the  Air  Force  and  sufficient  authority  and  institutional  knowledge  to 

coordinate  that  research 

•  better  quality  control  and  access  to  people  with  the  right  scientihc  expertise  and  resources 

•  increased  visibility  of  the  research  to  the  wider  Air  Force. 

Several  Air  Force  offices  continue  to  hnd  value  in  citing  the  work  and  have  requested  that 
it  be  made  part  of  the  public  record  for  current  and  future  reference.  The  remainder  of  this 
report,  therefore,  makes  public  those  findings  and  recommendations. 

Raymond  Conley 

Director,  Manpower,  Personnel,  and  Training  Program 

RAND  Project  AIR  FORCE 


Preface 


This  document  discusses  how  personnel  research  within  the  U.S.  Air  Force  supports  organiza¬ 
tional  policy  decisions.  It  discusses  the  history  of  personnel  research  in  the  Air  Force,  as  well  as 
the  present  personnel  research  efforts  and  the  organizations  that  house  them.  Though  the  Air 
Force  began  a  reorganization  that  affects  some  of  the  organizational  units  we  discuss  herein,  to 
the  extent  that  these  units  undertake  the  same  independent  personnel  research-related  activi¬ 
ties,  the  findings  still  pertain.  We  highlight  some  challenges  in  the  current  situation  and  the 
components  that  need  to  be  included  in  the  Air  Force’s  solution. 

The  research  reported  here  was  commissioned  by  the  U.S.  Air  Force  Directorate  of  Force 
Management  Policy  (AF/AIP)  and  conducted  within  the  Manpower,  Personnel,  and  Train¬ 
ing  Program  of  RAND  Project  AIR  FORCE  as  part  of  a  fiscal  year  2011  project,  “Enhancing 
Personnel  Selection  and  Screening  Methods.”  This  report  should  be  of  interest  to  those  set¬ 
ting  policy  for  personnel  research  efforts  or  conducting  research  on  personnel  issues  in  the  Air 
Force. 


RAND  Project  AIR  FORCE 

RAND  Project  AIR  FORCE  (PAF),  a  division  of  the  RAND  Corporation,  is  the  U.S.  Air 
Force’s  federally  funded  research  and  development  center  for  studies  and  analyses.  PAF  pro¬ 
vides  the  Air  Force  with  independent  analyses  of  policy  alternatives  affecting  the  development, 
employment,  combat  readiness,  and  support  of  current  and  future  air,  space,  and  cyber  forces. 
Research  is  performed  in  four  programs:  Force  Modernization  and  Employment;  Manpower, 
Personnel,  and  Training;  Resource  Management;  and  Strategy  and  Doctrine.  The  research 
reported  here  was  prepared  under  contract  FA7014-06-C-0001. 

Additional  information  about  PAF  is  available  on  our  website: 
http:  //  WWW.  rand,  org/paf/ 
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Summary 


In  1991,  the  Air  Force  Human  Resources  Laboratory  (AFHRL)  was  disestablished,  leaving 
some  of  its  functions  to  be  carried  out  by  various  organizations  across  the  Air  Force.  Similarly, 
the  Air  Force  Occupational  Measurement  Squadron  (AFOMS),  the  organization  responsible 
for  job  analysis  (JA) — known  in  the  Air  Force  as  occupational  analysis  (OA) — which  serves  as 
the  foundation  for  much  of  human  resource  management  (HRM),  has  been  downsized  sev¬ 
eral  times  in  the  past  three  decades,  with  the  latest  restructuring  and  downsizing  occurring  in 
2009.  Although  the  Air  Force  may  still  be  addressing  many  personnel  research  needs  through 
dispersed  organizations,  there  is  concern  that  the  decentralization  has  left  many  parts  of  the 
HRM  system  without  knowledge  of,  or  access  to,  supporting  personnel  research. 

Given  this  concern,  the  U.S.  Air  Force  Directorate  of  Force  Management  Policy  (AF/AIP) 
asked  RAND  Project  AIR  FORCE  (PAF)  to  assess  the  current  state  of  personnel  research  in 
the  Air  Force  and  examine  how  well  the  organizational  structure  underlying  the  existing  per¬ 
sonnel  research  efforts  is  meeting  the  broader  Air  Force’s  personnel  research  needs.  To  address 
this  concern,  we  focused  on  three  objectives:  (1)  describe  the  Air  Force  organizations  collect¬ 
ing  personnel-related  data  and  conducting  personnel-related  research,  identifying  the  type  of 
data  collected,  type  of  research  conducted,  and  how  these  initiatives  fit  into  the  organization’s 
mission;  (2)  examine  how  much  these  organizations  communicate  and  coordinate  their  efforts, 
share  data,  potentially  overlap  in  their  current  work,  and  have  the  necessary  resource  capac¬ 
ity  and  expertise;  and  (3)  identify  potential  gaps  in  the  structure  of  current  personnel  research 
efforts  and  recommend  strategies  for  eliminating  those  gaps.  To  address  these  study  objectives, 
we  reviewed  existing  documents  outlining  the  responsibilities  of  each  organization  (e.g..  Air 
Force  instructions  and  other  related  Air  Force  documentation  sources)  and  conducted  explor¬ 
atory  semistructured  interviews  with  representatives  at  organizations  we  identified  as  actively 
involved  in  personnel  research. 


The  Role  of  Personnel  Research  in  Organizations 

Efforts  to  determine  how  to  reduce  or  expand  training,  increase  motivation,  add  new  selection 
tests,  reduce  training  attrition,  and  anticipate  manning  requirements  could  all  be  considered 
personnel  research.  Such  personnel  research  informs  such  areas  as  recruitment,  selection,  clas¬ 
sification,  training,  performance  management,  and  work  attitudes,  which  interact  through  an 
employee’s  tenure  on  the  job.  For  example,  recruitment  should  focus  on  individuals  with  the 
best  fit  between  person  and  organization,  in  terms  of  values  and  knowledge,  skills,  abilities, 
and  other  characteristics  (KSAOs),  and  selection  should  further  narrow  the  field  to  allow  entry 
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only  to  those  employees  with  the  best  potential  to  facilitate  the  organization’s  goals.  Perfor¬ 
mance  management  over  an  employee’s  tenure  facilitates  cohesive  employee  development  and 
provides  a  way  to  reinforce  desired  behaviors.  Attention  to  work  attitudes  throughout  can  facil¬ 
itate  transfer  of  training,  performance  of  valued  behaviors,  and  organizational  change  efforts. 

The  structure  and  synergy  of  these  HRM  practices  are  best  informed  by  well-executed 
data — ^JA  data,  attitudinal  data,  and  administrative  data — gathered  through  surveys,  inter¬ 
views,  focus  groups,  observation,  tests,  and  human  resource  recordkeeping.  Ideally,  these 
data  are  recorded  regularly  and  longitudinally  to  better  model  organizational  experience  and 
change.  Aligning  the  HRM  system  as  a  whole  facilitates  the  attainment  of  strategic  goals  and 
enables  an  organization  to  establish  a  strong,  consistent  climate.  Without  information  about 
employee  experiences  on  the  job,  an  organization  may  blindly  grope  its  way  along  and  bypass 
the  advantage  of  data-driven  HRM  decisions.  This  basis  in  data  is  particularly  important  in  a 
time  of  resource  constraints;  the  literature  demonstrates  the  complexity  that  inheres  in  restruc¬ 
turing  and  downsizing:  Mistakes  may  be  costly.  This  type  of  forward-looking  strategic  person¬ 
nel  research  has  not  been  a  priority  in  the  Air  Force  for  some  time,  as  exemplihed  by  the  dises¬ 
tablishment  of  organizations  and  deprioritization  of  funding  devoted  to  the  purpose. 


A  Very  Brief  History  of  the  Management  of  Personnel  Research  in  the  Air 
Force 

AFHRL  historically  was  tasked  with  responsibility  for  personnel  research  and  development. 
With  AFHRL  and  its  heirs  gone,  no  organization  has  had  that  same  level  of  responsibility  for 
research  and  the  broad,  strategic  research  and  development  focus  for  many  years.  Nevertheless, 
OA  data  continue  to  be  collected  and  utilized  for  purposes  of  informing  change  in  tech  train¬ 
ing  and  designing  Weighted  Airman  Promotion  System  (WAPS)  tests,  and  other  organizations 
accommodate  requests  to  conduct  personnel  research  or  have  taken  on  responsibility  for  spe¬ 
cific  aspects  of  personnel  research.  Although  some  research  and  the  collection  of  a  variety  of 
personnel  data  have  continued,  there  is  no  single  resource  for  consumers  of  personnel  research. 
The  deprioritization  and  dispersion  of  personnel  research  endeavors,  and  the  current  lacuna  in 
strategic  high-risk  personnel  research  and  development,  have  led  to  a  variety  of  issues. 


Personnel  Research  and  Data  Collection  in  the  Air  Force  Today 

Personnel  research  efforts  are  currently  decentralized,  with  several  different  organizations 
involved  in  collecting  personnel-related  data  and  conducting  personnel-related  research.  These 
organizations  include  both  internal  Air  Force  organizations  and  external  contractors.  The  data 
being  collected  by  these  organizations  range  from  JA  data  to  test  scores,  performance  ratings, 
and  data  on  various  workplace  attitudes.  Research  efforts  range  from  descriptive  analyses  of 
personnel  data  to  longitudinal  studies,  such  as  those  looking  at  test  validation.  Together,  these 
organizations  are  collecting  much  of  the  key  data  required  for  current  Air  Force  needs  and 
engaging  in  important  personnel-related  research. 


Summary  xv 


How  Well  the  Current  Organizational  Structure  of  Personnel  Research 
Efforts  Meets  Air  Force  Needs 

Although  many  of  the  key  elements  of  a  smoothly  working  personnel  research  system  exist 
in  the  Air  Force  or  can  he  brought  to  hear  with  help  from  outside  contractors,  the  current 
system  is  not  optimal.  Specifically,  we  identified  some  critical  issues  that  inhibit  the  quality 
and  efficiency  of  current  personnel  research  efforts;  these  include  narrow  organizational  mis¬ 
sions,  inconsistent  data-collection  coordination  and  data  sharing,  a  lack  of  internal  person¬ 
nel  research  expertise,  limited  resources,  reliance  on  contractors,  and  potential  duplication  of 
effort.  To  enhance  the  quality  and  efficiency  of  the  system,  we  identified  several  needed  com¬ 
ponents  for  organizational  change. 

First,  there  is  a  need  to  put  in  place  an  organizational  structure  that  has  clear  oversight 
responsibility  over  all  the  personnel  research  efforts  ongoing  in  the  Air  Force.  Although  the  Air 
Force  Manpower  Agency  (AFMA)  is  the  hub  for  Air  Force-wide  attitude  surveys  and  other 
organizations  serve  as  the  hubs  for  various  other  types  of  research  (e.g.,  AF/AIP  serves  as  the 
hub  for  selection  and  testing  research),  no  one  organization  or  structure  actively  coordinates 
them  all.  A  key  aspect  of  this  component  would  be  appropriate  funding  to  enable  a  strategic, 
long-term  research  and  development  function. 

Second,  to  achieve  the  benefits  of  such  oversight,  the  organizational  structure  must  have 
sufficient  authority  to  coordinate  the  disparate  elements  of  the  Air  Force’s  existing  personnel 
research  system.  Knowing  what  data  are  available  to  Air  Force  decisionmakers  is  one  step;  the 
next  step  is  having  the  authority  to  execute  actions  to  eliminate  duplication  of  effort  for  great¬ 
est  efficiency,  require  the  disparate  elements  to  communicate  as  needed,  and  institute  new 
research  or  collection  of  new  data  elements  where  gaps  in  the  requisite  data  currently  exist.  The 
structure  must  also  have  sufficient  authority  and  legitimacy  to  request  the  existing  data  from 
the  collecting  organizations  to  inform  any  decisionmaking  for  the  personnel  system  as  a  whole. 

Third,  the  organizational  element  must  include  institutional  knowledge — a  deep  under¬ 
standing  of  the  Air  Force  and  the  way  things  are  customarily  done  (its  culture,  values,  and,  of 
course,  its  personnel  system) — to  help  determine  true  gaps  in  knowledge  and  the  collection  of 
the  most-ideal  data  to  fill  these  gaps. 

A  fourth  component  is  quality  control  to  ensure  that  ongoing  and  one-off  research  efforts 
meet  minimum  standards  for  quality  and  utility.  A  quality  control  function  ensures  that  col¬ 
lected  data  can  be  more  easily  integrated  into  a  smoothly  functioning  system  and  that  a  ques¬ 
tion  does  not  remain  unanswered  because  data  collected  for  one  or  more  aspects  of  the  problem 
are  inadequate  to  meet  the  need. 

A  fifth  component  is  access  to  scientific  expertise  and  resources — sufficient  resources  in 
terms  of  both  funding  and  expertise  to  enable  the  Air  Force  to  optimize  its  personnel  system 
and  operate  as  efficiently  as  possible  without  overburdening  existing  or  new  organizational 
structures.  Without  expertise  and  other  resources,  quality  control  is  just  a  compelling  slogan 
rather  than  an  operationalized  component. 

Providing  wider  data  availability  is  another  key  component.  Currently,  no  organization’s 
primary  mission  is  to  share  all  collected  data  with  others,  and,  in  supporting  primary  missions, 
the  priority  status  given  to  doing  the  basic  work  of  sharing  data  resources  is  understandably 
low.  A  good  solution  would  enable  data  to  be  shared  more  efficiently  without  upsetting  other 
vital  priorities  and  missions.  Additional  personnel  to  organize  the  effort  and  lighten  the  load 
on  organizations  by  taking  on  such  “nonessential”  tasks  as  formatting  data  sets  for  sharing 
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(including  alleviating  concerns  about  confidentiality  of  the  data,  and  ensuring  security)  would 
facilitate  the  data-sharing  process. 

A  final  component  is  providing  increased  visibility  to  the  wider  Air  Force.  Visibility  would 
be  key  to  enabling  the  organizational  structure  to  serve  as  a  clearinghouse  because  it  is  prob¬ 
able  that  there  are  ongoing  projects  undertaken  by  organizations  with  relatively  little  formal 
connection  to  the  Air  Force  personnel  research  system  that  are  engaging  in  relevant  efforts. 
This  visibility  component  may  also  imply  that  some  marketing  of  organizational  purpose  and 
resources  would  need  to  be  done  on  an  ongoing  basis  on  behalf  of  the  organizational  structure. 

How  such  key  components  are  operationalized  depends  on  the  organizational  structure 
chosen  by  the  Air  Force  to  bring  the  personnel  research  system  into  alignment.  Further,  cer¬ 
tain  issues  require  consideration.  As  in  project  management’s  concept  of  the  triple  constraint 
(projects  can  be  economical,  high  quality,  or  fast,  but  one  can  achieve  only  two  out  of  three; 
e.g.,  Kernzer,  2009),  trade-offs  must  be  made.  With  this  in  mind,  we  describe  three  potential 
scenarios  for  these  organizations  going  forward  (a  baseline  plus  one  scenario,  a  comprehensive 
alignment  scenario,  and  a  hybrid  scenario).  These  scenarios  incorporate  all  the  key  components 
and  demonstrate  various  ways  they  may  be  incorporated  in  order  to  provide  the  solution  to  the 
current  gaps  (narrow  organizational  missions,  inconsistent  data- collection  coordination  or  data 
sharing,  a  lack  of  internal  personnel  research  expertise,  limited  resources,  reliance  on  contrac¬ 
tors,  and  potential  duplication  of  effort).  In  all  cases,  we  recommend  inclusion  of  a  Major  Force 
Program  6  funding  line  to  enable  the  organizational  structure  to  take  on  a  strategic  research 
and  development  focus  and  be  proactive  rather  than  reactive.  In  the  current  climate  of  fiscal 
austerity,  force  restructuring  should  be  undertaken  with  the  support  of  personnel  research. 

First,  these  organizations  may  continue  as  at  present,  in  their  existing  hierarchies,  with  their 
current  missions  and  functions  (the  baseline  plus  one  scenario).  The  oversight  organizational 
structure  could  be  relatively  leanly  composed  of  a  few  key  personnel  to  provide  for  the  vari¬ 
ous  functions,  including  strategic  research  and  development.  A  small  oversight  organization  is 
relatively  economical,  involves  the  least  wide-reaching  change,  and  will  least  threaten  existing 
stakeholders  and  stovepipes.  Although  this  means  that  it  may  be  easier  and  faster  to  imple¬ 
ment,  it  also  means  that  success  may  not  be  as  wide-ranging  and  it  may  be  problematic  to 
structure  the  organization  with  sufficient  personnel  to  enforce  the  overall  clearinghouse  man¬ 
date  and  achieve  effectiveness  and  strategic  alignment  of  personnel  research.  In  terms  of  the 
triple  constraints,  this  course  maximizes  speed  and  economics  rather  than  high  quality. 

A  second  alternative  is  that  the  various  personnel  research  organizations  discussed  in  this 
document  could  be  reorganized  under  a  single  oversight  organization  (the  comprehensive  align¬ 
ment  scenario).  This  places  all  the  stakeholders  together  organizationally  and  emphasizes  the 
commonalities  rather  than  the  differences,  which  would  ideally  facilitate  communication  and 
coordination;  it  would  also  create  an  organization  with  an  explicit  responsibility  for  align¬ 
ing  the  personnel  system  with  the  research  that  should  support  decisionmaking.  However,  it 
would  incur  costs  in  both  time  and  money  to  move  and  integrate  organizations,  as  well  as  to 
recruit  needed  expertise,  both  for  research  support  and  for  a  marketing/liaison  element.  The 
old  AFHRL  was  criticized  for  a  lack  of  responsiveness  to  managers  on  the  ground  and  in  the 
field  and  to  policymakers  themselves.  A  new  personnel  research  organization  would  need  to 
remain  integrated  with  the  mission  of  the  Air  Force  as  a  whole,  inclusive  of  emergent  concerns, 
to  forestall  this  issue.  This  course  of  action  maximizes  quality  and  would  likely  not  be  either 
economical  or  fast. 
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A  third  option,  a  hybrid  approach  (the  hybrid  scenario),  could  also  make  sense.  A  new 
Personnel  Research  Directorate  with  a  division  focused  more  narrowly  on  JA-type  data  collec¬ 
tion  could  house  the  organization  known  as  the  Occupational  Analysis  Division  (technically, 
the  OA  flight)*  and  the  Management  Engineering  Division  of  AFMA,  and  a  second  director¬ 
ate  could  incorporate  the  other,  more- disparate  elements  of  AFMA,  the  organization  known 
as  the  Airman  Advancement  Division  (technically  called  the  Test  and  Evaluation  flight), ^  and 
the  Data  Reports  and  Retrievals  Branch,  Analysis  Branch,  and  the  Research  and  Assessments 
Branch  currently  at  the  Air  Force  Personnel  Center  (AFPC).  Benefits  include  the  advantages 
that  accrue  from  collocating  (organizationally  if  not  geographically)  organizations  that  have 
personnel  research  as  a  shared  activity.  However,  merely  collocating  these  organizations  would 
not  negate  the  need  to  hire  additional  personnel  to  coordinate  and  communicate  among  the 
organizations  and  develop  and  execute  a  strategic  personnel  research  plan;  in  some  senses, 
the  needs  would  he  more  acute  because  the  existing  organizations,  maintained  in  the  overall 
structure,  have  historically  been  stovepiped  in  their  roles  and  objectives.  A  strategic  research 
element  should  be  located  with  the  rearranged  organizations  in  San  Antonio  to  fully  optimize 
the  geographic  advantage,  but  some  linkage  to  the  strategic  training  research  conducted  in  the 
711th  Human  Performance  Wing  (711  HPW)  is  advisable  and  necessary  to  bring  the  entirety 
of  the  system  into  true  alignment.  This  course  does  not  explicitly  optimize — or  sacrifice — cost, 
speed,  or  quality.  However,  this  course  would  move  the  Air  Force  closer  to  an  optimization 
of  the  personnel  research  system  and  does  have  compelling  benefits  in  terms  of  convenience. 

Ultimately,  bringing  the  personnel  research  system  of  the  Air  Force  into  alignment  is  an 
exercise  in  organizational  change.  Even  the  most  economical  and  quick  change  the  Air  Force 
might  make  would  require  sustained  commitment  from  leadership  at  the  highest  levels.  How¬ 
ever,  given  the  costs  that  inhere  in  the  personnel  system  itself,  aligning  the  personnel  research 
system  to  better  support  strategic  decisionmaking  offers  the  potential  for  large  dividends,  par¬ 
ticularly  in  a  time  of  resource  constraints. 


*  Currently,  the  Occupational  Analysis  Division  is  officially  designated  as  flight  within  a  squadron.  See  Chapter  Three  for 
more  explanation. 

^  Currently,  the  Airman  Advancement  Division  is  officially  designated  as  flight  w^ithin  a  squadron.  See  Chapter  Three  for 
more  explanation. 
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Introduction 


Background 

According  to  the  2010  Quadrennial  Defense  Review  (QDR),  “America’s  men  and  women  in 
uniform  constitute  the  Department’s  most  important  resource”  (p.  49).  Thus,  a  well-designed 
human  resource  management  (HRM)  system' — one  that  selects  the  best  people,  provides  the 
best  training,  puts  the  right  people  in  the  right  jobs,  and  successfully  promotes  and  retains 
the  highest-quality  performers — is  vital  to  the  U.S.  Air  Force’s  success.  In  the  past  several 
decades,  the  Air  Force  has  implemented  various  organizational  downsizings,  office  closures, 
and  changes  in  organizational  structures  that  have  resulted  in  decentralizing  how  it  conducts 
personnel  research.  One  of  the  most  notable  changes  occurred  in  1998,  when  the  last  organi¬ 
zational  unit  with  broad  responsibility  for  personnel  research  and  development,  the  Air  Force 
Research  Laboratory’s  (AFRL’s)  Human  Resources  Directorate,  was  disestablished,  leaving 
some  of  its  functions  to  be  carried  out  by  various  organizations  across  the  Air  Force,  but  none 
with  such  a  broad  strategic  research  mandate.  This  was  the  culmination  of  a  series  of  lab  merg¬ 
ers  and  reorganizations  that  began  (for  personnel  research)  in  1991  with  the  disestablishment 
of  the  Air  Force  Human  Resources  Laboratory  (AFHRL).  Similarly,  the  Air  Force  Occupa¬ 
tional  Measurement  Squadron  (AFOMS),  the  organization  responsible  for  job  analysis  (JA) 
(known  in  the  Air  Force  as  occupational  analysis  [OA]),  which  serves  as  the  foundation  for 
much  of  HRM,  has  been  downsized  several  times  in  the  past  three  decades,  with  the  latest 
restructuring  and  downsizing  occurring  in  2009. 

With  no  central  organization  in  charge  of  personnel  research  efforts,  and  in  particular  the 
kinds  of  broadly  focused  strategic  research  and  development  that  were  the  purview  of  AFHRL 
and  its  heirs,  disparate  organizations  across  the  Air  Force — such  as  the  Air  Force  Deputy 
Chief  of  Staff,  Manpower  and  Personnel  (AF/Al);  the  Air  Force  Deputy  Chief  of  Staff,  Stud¬ 
ies  and  Analyses,  Assessments  and  Lessons  Learned  (AF/A9);  the  Air  Force  Manpower  Agency 
(AFMA);  the  Air  Force  Personnel  Center  (AFPC);  the  Studies  and  Analysis  Squadron  (SAS); 
and  components  of  AFRL — must  conduct  or  sponsor  their  own  individual  studies  to  address 
personnel-related  issues  as  the  perceived  need  arises.  Thus,  the  focus  tends  to  be  more  reac¬ 
tive  than  proactive,  and  programmatic  research  is  difficult  if  not  impossible.  Although  the  Air 
Force  may  still  be  addressing  many  personnel  research  needs  through  these  dispersed  organi¬ 
zations,  there  is  concern  that  the  decentralization  and  deprioritization  of  the  broad  personnel 
research  and  development  focus  have  left  many  parts  of  the  HRM  system  without  knowledge 


'  Although  there  is  no  standard  definition  of  these  terms,  we  use  the  term  HRM  broadly  to  include  any  effort  directed 
at  improving  the  effectiveness  or  efficiency  of  personnel  or  at  reducing  personnel  costs  and  personnel  research  to  mean  any 
systematic  investigation  into  how  to  improve  some  aspect  of  HRM. 
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of,  or  access  to,  the  requisite  personnel  research.  For  that  reason,  the  Air  Force  Directorate  of 
Force  Management  Policy  (AF/AIP)  has  questioned  whether  the  current  structure  of  personnel 
research  efforts  could  he  improved  to  more  effectively  and  more  efhciently  meet  the  Air  Force’s 
personnel  research  needs. 


Study  Objectives  and  Analytical  Approach 

Given  this  concern,  AF/AlP  asked  RAND  Project  AIR  FORCE  (PAF)  to  assess  the  current 
state  of  personnel  research  in  the  Air  Force  and  examine  how  much  the  organizational  struc¬ 
ture  underlying  the  existing  personnel  research  efforts  is  meeting  the  Air  Force’s  broader  per¬ 
sonnel  research  needs.  To  address  this  concern,  we  focused  on  three  main  objectives: 

1.  Describe  the  Air  Force  organizations  collecting  personnel-related  data  and  conducting 
personnel-related  research.  For  each,  identify  the  type  of  data  collected,  type  of  research 
conducted,  and  how  these  initiatives  fit  into  the  organization’s  mission. 

2.  Examine  how  much  these  organizations  communicate  and  coordinate  their  efforts, 
share  data,  potentially  overlap  in  their  current  work,  and  have  the  necessary  resource 
capacity  and  expertise. 

3.  Identify  potential  gaps  in  the  structure  of  current  personnel  research  efforts  and  recom¬ 
mend  strategies  for  eliminating  those  gaps  to  ensure  that  the  Air  force’s  broader  person¬ 
nel  research  needs  are  being  met. 

To  address  these  three  study  objectives,  we  reviewed  existing  documents  outlining  the 
responsibilities  of  each  organization  (e.g..  Air  force  instructions  and  other  related  Air  force 
documentation  sources)  and  conducted  exploratory  interviews  with  representatives  at  organi¬ 
zations  we  identified  as  actively  involved  in  personnel  research.  We  started  our  sample  by  inter¬ 
viewing  former  members  of  Af  HRf  and  current  members  of  the  newly  downsized  Af  OMS 
because  these  two  organizations  have,  at  least  in  the  past,  served  as  the  Air  force’s  main  source 
of  personnel  research  and  the  OA  data  that  can  serve  as  a  cornerstone  of  such  work.^  We  then 
used  a  snowball  sampling  technique  and  our  knowledge  of  Air  force  personnel  research  efforts 
to  identify  other  organizations  involved  in  personnel  research  in  the  Air  force. ^ 

from  November  2009  to  July  2010,  we  conducted  several  exploratory  interviews  with  cur¬ 
rent  and  retired  staff  from  12  different  Air  force  organizations,  including  Af  HRf ,  AfOMS, 
the  two  flights  within  Air  Education  and  Training  Command  (AETC)  known  as  the  Occu¬ 
pational  Analysis  Division  (OAD)  and  the  Airman  Advancement  Division  (AAD),"^  AfPC, 
AfMA,  AETC’s  SAS,  and  several  Af/Al  directorates.  Although  this  coverage  is  not  compre¬ 
hensive  (and,  in  fact,  comprehensive  coverage  was  beyond  the  scope  of  this  paper),  we  do  feel 


^  The  importance  of  this  type  of  data  is  explained  in  detail  in  Chapter  Two. 

^  Snowball  sampling  is  a  technique  of  using  new  information  collected  during  a  project  to  continuously  build  on  an  exist¬ 
ing  sample  throughout  the  project.  In  our  case,  during  all  of  our  interviews  and  discussions  with  those  involved  in  current 
personnel  research  efforts,  we  asked  whether  there  were  any  additional  organizations  we  should  consider  in  our  study.  Sev¬ 
eral  of  the  organizations  included  in  our  study  were  identified  in  this  way. 

^  Although  these  organizations  have  included  the  word  division  in  their  titles,  they  are  officially  considered  flights,  not 
divisions.  See  Chapter  Three  for  more  explanation. 
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we  identified  many  of  the  major  players  in  the  personnel  research  arena  in  this  manner  and 
gained  a  hroad  perspective  on  ongoing  endeavors  suitable  for  our  policy  objectives.  The  inter¬ 
views  lasted  between  one  and  two  hours  and  covered  such  topics  as^ 

•  respondents’  background 

•  organization’s  past  and  current  personnel  research  and  data- collection  activities 

•  potential  and  actual  clients  and  users  of  the  data  they  collect 

•  internal  expertise  and  resources  related  to  conducting  personnel  research 

•  use  of  contractors  to  outsource  personnel  research 

•  awareness  of  other  existing  personnel  data-collection  efforts 

•  involvement  in  any  data  sharing  with  other  organizations. 

During  the  interviews,  we  also  asked  the  organizational  representatives  to  provide  any 
official  or  unofficial  documents  that  described  their  organization’s  activities  and  talk  about 
the  processes  used  to  collect  and  analyze  data,  the  products  produced,  or  any  request  they 
had  received  for  assistance  from  other  organizations.  In  addition,  we  examined  the  Air  Force 
portal,  which  may  be  thought  of  as  the  Air  Force’s  own  worldwide  intranet,  and  related  web¬ 
sites  for  official  documents  (such  as  Air  Force  instructions  and  fact  sheets)  describing  an  orga¬ 
nization’s  responsibilities. 

Using  the  interview  results,  we  discuss  how  personnel  research  hts  into  each  organiza¬ 
tion’s  primary  mission,  how  much  the  organizations  have  staff  with  personnel  research  exper¬ 
tise,  the  data-collection  coordination  and  sharing  among  organizations,  and  potential  dupli¬ 
cation  of  effort.  Using  our  evaluation  of  these  key  factors,  we  assessed  how  well  the  current, 
decentralized  organizational  structure  of  personnel  research  is  meeting  the  broader  needs  of 
the  Air  Force.  Using  the  information  amassed,  we  identihed  how  personnel  research  activities 
could  potentially  be  managed  more  efficiently  or  effectively. 


Report  Timing 

As  noted  in  the  foreword  and  in  this  chapter,  the  research  documented  here  was  carried  out 
primarily  in  2009  through  2011,  and  some  reorganization  has  occurred  subsequently  (for 
example,  AFMA  and  AFPC  have  been  merged  into  a  single  field  operating  agency,  or  FOA). 
However,  to  our  knowledge,  alignment  of  personnel  research  was  not  a  consideration  in  the 
reorganization,  and  disconnects  are  still  of  concern.  The  recommendations  discussed  in  the 
report  (such  as  increasing  alignment,  communication,  data  sharing,  and  dissemination  of  per¬ 
sonnel  research  findings)  are  therefore  still  potentially  applicable  to  the  new  organizations  and 
can  still  be  used  to  help  guide  further  reorganization  to  increase  efficiencies  and  reduce  costs. 
The  chapters  that  follow  document  the  state  of  personnel  research  in  the  Air  Force  at  the  time 
of  our  research,  but,  most  importantly,  they  highlight  and  illustrate  several  obstacles  to  meet¬ 
ing  the  Air  Force’s  personnel  research  needs  that  may  still  remain. 


^  In  some  interviews,  not  all  topics  were  covered. 
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Organization  of  This  Report 

In  Chapter  Two,  we  review  key  components  of  a  successful  personnel  research  program.  In 
Chapter  Three,  we  provide  a  historical  review  of  personnel  research  within  the  Air  Force, 
highlighting  the  work  of  AFHRL.  In  Chapter  Four,  we  provide  descriptions  of  other  organiza¬ 
tions  involved  in  personnel  data  collection  and  provide  an  overview  of  their  personnel  research 
activities.  In  Chapter  Five,  we  describe  some  of  the  problems  associated  with  the  current  orga¬ 
nization  of  personnel  research  in  the  Air  Force;  and  finally,  in  Chapter  Six,  we  provide  recom¬ 
mendations  regarding  ways  to  improve  it. 


CHAPTER  TWO 


The  Role  of  Personnel  Research  in  Organizations 


People  are  among  the  most  valuable  and  expensive  resources  in  organizations  (Cascio,  2003; 
Lawler  and  Boudreau,  2009;  Ployhart,  2006).  Given  this,  both  academics  and  organizations 
have  a  great  deal  of  interest  in  personnel,  and  several  academic  disciplines  have  produced  nearly 
a  century  of  research  on  the  topic.'  Taken  as  a  whole,  this  research  has  led  to  a  set  of  key  fea¬ 
tures  that  define  a  good  HRM  system. 

This  chapter  provides  an  overview  of  some  of  those  features,  focusing  on  two  issues  rel¬ 
evant  to  this  study:  reliance  on  personnel  research  to  drive  HRM  decisions,  and  the  role  of 
different  types  of  data  in  HRM. 


Defining  Human  Resource  Management  and  Personnel  Research 

HRM  and  personnel  research  can  mean  different  things  to  different  people.  Regardless  of  these 
differences,  we  use  the  term  broadly  to  refer  to  any  effort  directed  at  improving  the  effec¬ 

tiveness  or  efficiency  of  personnel  or  at  reducing  personnel  costs  and  personnel  research  to  mean 
any  systematic  investigation  into  how  to  improve  some  aspect  of  HRM.^  In  this  sense,  efforts 
to  determine  how  to  reduce  or  expand  training,  increase  motivation,  add  new  selection  tests, 
reduce  training  attrition,  and  anticipate  manning  requirements  could  all  be  considered  person¬ 
nel  research.  Table  2.1  provides  the  wide  variety  of  HRM  topic  areas  that  can  cover  the  entire 
life  cycle  of  an  employee  and  examples  of  narrow  personnel  research  objectives  that  fall  within 
each  topic  area.  Some  research  areas  that  we  would  not  consider  in  our  definition  fall  under  the 
rubric  of  human  factors  (i.e.,  human  interaction  with  machines  and  equipment),  focus  more 
on  macro  perspectives  rather  than  individual-level  components  (i.e.,  some  subfields  in  organi¬ 
zational  behavior),  or  may  be  classified  as  research  unrelated  to  the  job  or  work  context. 


Aligning  the  Components  of  Human  Resource  Management 

Although  many  organizations  often  treat  personnel  practices,  such  as  recruiting,  selecting,  and 
training  personnel,  as  separate  functions,  personnel  research  scientists  argue  that  these  HRM 


'  Examples  of  disciplines  contributing  to  the  study  of  personnel  management  include  industrial  and  organizational  (I/O) 
psychology,  social  psychology,  sociology,  organizational  behavior,  industrial  relations,  and  HRM. 

^  These  are  not  terms  that  have  a  standard  meaning  or  for  which  there  is  a  common  definition  in  the  academic  or  scientific 
literature;  other  valid  definitions  of  these  terms  may  exist.  We  chose  these  terms  to  describe  the  factors  and  facets  of  research 
that  are  the  focus  of  this  paper. 
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Table  2.1 

Examples  of  Human  Resource  Management  Topic  Areas  and  Personnel  Research  Objectives 


Example  of  HRM  Topic 
Area 

Example  of  Personnel  Research  Objective 

Recruiting 

Identify  types  of  candidates  to  target  for  recruiting 

Selection 

Decide  which  tests  to  use  to  screen  out  less  successful  officer  candidates 

Decide  how  to  screen  people  for  promotions 

Training 

Develop  training  to  improve  skills  upon  entry  to  the  organization  or  to  a  particular 
career  field 

Evaluate  the  success  of  training 

Performance  appraisal 

Justify  termination  of  unproductive  personnel 

Identify  the  top  candidates  for  promotion 

Identify  personnel  deserving  of  bonuses,  pay  raises,  recognition,  or  other  rewards 

Employee  development 

Establish  the  criteria  for  giving  employees  feedback  on  areas  for  improvement 

Design  education  and  development  to  improve  skills  across  a  career 

Interests  and  attitudes 

Measure  job  satisfaction 

Change  perceived  fairness  of  an  organizational  policy 

Culture  or  climate 

Increase  workplace  safety  behaviors 

Encourage  application  of  training  on  the  job 

Increase  unit  cohesion 

Evaluate  the  effectiveness  of  sexual  harassment  training 

Retention 

Reduce  training  attrition 

Retain  the  best  officers 

Determine  a  need  for  retention  bonuses 

Motivation 

Identify  how  best  to  motivate  employees 

Identify  how  to  retain  the  best  officers 

Identify  why  employees  are  engaging  in  counterproductive  work  behaviors 

Job  classification 

Decide  whether  two  career  fields  are  similar  enough  that  they  should  be  grouped 
together  into  one  career  field 

Manpower  planning 

Identify  how  many  people  are  needed  in  a  particular  job 

NOTE:  This  is  not  an  exhaustive  list  of  HRM  topic  areas  or  objectives. 


functions  should  be  instead  managed  as  a  cohesive,  planful,  and  integrated  system  of  interlock¬ 
ing  parts — parts  that  should  be  aligned  for  the  strongest  effect  not  only  with  each  other  but 
also  with  the  strategic  purpose  of  the  overall  organization  (P.  Wright  and  McMahan,  1992; 
Lepak  et  ah,  2006).  This  view  is  sometimes  referred  to  as  strategic  HRM. 

An  integrated  system  ensures  that  one  part  of  the  system  does  not  inadvertently  conflict 
with  the  other  parts  of  the  system.  For  example,  Bowen  and  Ostroff  (2004;  see  also  Ployhart, 
2006)  note  that,  if  all  aspects  of  the  HRM  system  are  working  together  to  accomplish  the 
same  goals,  this  consistency  sends  a  coherent  and  cohesive  message  to  employees,  which  leads 
to  establishing  strong  organizational  norms  in  an  organization.  These  strong  organizational 
norms  can,  in  turn,  help  strengthen  the  relationship  between  individual-level  HRM  efforts  and 
macro-level  organizational  goals,  thus  helping  to  ensure  that  employees  know  which  aspects  of 
behavior  will  be  rewarded  and  punished.  This  knowledge  ultimately  leads  to  more  individuals 
behaving  in  ways  that,  in  the  aggregate,  are  beneficial  to  organizational  performance.  In  con¬ 
trast,  if  various  HRM  functions  are  not  well-coordinated,  they  may  inadvertently  send  incon¬ 
sistent  or  even  contradictory  messages  about  what  is  valued  by  the  organization.  In  such  cases. 
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the  resulting  effort  would  be  ineffective  at  creating  the  desired  change  and  would  have  a  much 
smaller  potential  impact  on  the  organization’s  overall  success. 

Such  poorly  coordinated  efforts  or  inconsistent  messages  in  an  HRM  system  can  also 
negatively  affect  employee  attitudes  about  the  organization.  There  is  a  wealth  of  research  show¬ 
ing  that  inconsistent  policies,  or  policies  viewed  as  procedurally  unfair  or  unjust,  can  lead 
to  lower  organizational  commitment  and  job  performance  and  increase  organizational  with¬ 
drawal.  (For  a  recent  summary,  see  Colquitt  et  ah,  2001.) 

But,  to  send  a  coherent  message,  it  is  necessary  to  have  a  structure  that  fosters  coordi¬ 
nation  and  integration  among  the  different  aspects  of  the  system  and  a  consistent  focus  on 
the  central  strategic  goals  of  the  organization.  One  example  of  this  synergy  is  that  all  aspects 
of  the  system  should  emphasize  improving  job  performance:  Personnel  recruitment  should 
target  those  likely  to  be  the  most-effective  personnel  for  the  job  requirements  the  organiza¬ 
tion  demands,  selections  from  this  recruitment  pool  should  target  those  most  able  to  fulfill 
job  requirements,  training  should  be  tailored  to  enable  personnel  to  do  their  required  job  tasks 
well,  promotions  should  reward  the  most  capable,  compensation  (monetary  and  otherwise) 
should  enable  the  organization  to  retain  the  best  and  brightest,  and  the  organization  should 
have  policies  in  place  to  encourage  employees  whose  performance  is  inadequate  to  move  on  (or 
enable  the  organization  to  push  them  out  directly).^  In  this  example,  if  the  individual  compo¬ 
nents  of  the  system  do  not  have  some  overarching  coordination  and  communication,  there  is 
no  guarantee  that  all  aspects  of  the  HRM  system  will  share  the  same  focus  on  job  performance 
or  that,  even  when  they  do,  they  will  share  the  same  definition  of  the  job  performance  that 
each  is  hoping  to  improve. 

Regardless,  the  parts  will  ultimately  act  together  when  affecting  the  organization’s  overall 
success,  which  raises  the  question  of  how  much  of  an  impact  failure  to  coordinate  the  various 
individual  components  of  a  personnel  resource  system  has  on  the  success  of  the  whole  orga¬ 
nization.  Although  a  lack  of  coordination  can  likely  be  ignored  or  tolerated  without  causing  a 
catastrophe,  scientists  who  study  HRM  systems  agree  that,  when  the  individual  components 
are  not  carefully  and  strategically  aligned,  they  may  end  up  rewarding  or  promoting  behavior 
that  does  not  facilitate  overall  organizational  performance  (Bowen  and  Ostroff,  2004).  Such 
actions  are,  at  best,  inefficient  or  ineffective  practices  that  ultimately  hurt  the  organization’s 
bottom  line.  Note  that,  in  an  organization  with  a  large  personnel  system,  the  costs  of  small 
inefficiencies  can  accrue  dramatically. 


The  Three  Types  of  Data  Typically  Used  in  Personnel  Research 

Personnel  research,  the  bedrock  of  HRM,  relies  heavily  on  three  different  types  of  data:  admin¬ 
istrative  personnel  data,  attitudinal  or  opinion  data,  and  JA  data. 

Administrative  Personnel  Data 

Organizations  typically  collect  and  retain  administrative  data  on  employees,  such  as  selection 
test  scores,  hire  date,  pay  rate,  completed  training  courses,  and  attrition.  Often,  such  data 


^  Lepak  et  al.  (2006)  suggest  that  systems  may  have  one  or  more  of  three  policy  emphases:  employee  knowledge,  skills, 
abilities,  and  other  characteristics  (KSAOs);  management  of  employee  attitudes  and  motivation;  or  employee  opportunities 
to  contribute.  In  our  example  of  performance,  practices  are  naturally  integrated  among  these  three  policy  areas. 
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record  the  key  indicators  of  organizational  investment  in  an  employee  and  their  development. 
Because  such  data  are  so  accessible,  they  are  often  used  in  personnel  research;  however,  person¬ 
nel  records  can  he  limited  in  the  types  of  information  available.  For  this  reason,  additional  data 
are  often  needed  to  address  personnel  research  efforts  and,  ultimately,  identify  organizational 
issues  and  potential  solutions.  A  highly  effective  use  of  administrative  data  is  to  combine  them 
with  other  sources  of  data  to  reduce  data-collection  burden  and  ultimately  to  reduce  personnel 
research  costs. 

Altitudinal  and  Perceptual  Data 

Attitudes,  opinions,  beliefs  and  perceptions,  and  social  norms  can  explain  and  predict  how  and 
why  people  do  what  they  do  (see,  e.g.,  Ajzen,  1991,  2001;  Ajzen  and  Fishbein,  1980;  Bandura, 
1977;  Chaiken  and  Stangor,  1987;  Judge  et  ah,  2001;  Roznowski  and  Hulin,  1992).  For  this 
reason,  surveys,  interviews,  and  focus  groups  commonly  ask  questions  about  them,  and  the 
resulting  data  can  be  used  to  diagnose  areas  in  need  of  improvement  or  to  measure  whether 
efforts  to  change  attitudes,  norms,  or  beliefs  have  been  effective.  This  type  of  data  is  used  to 
inform  many  aspects  of  personnel  research,  and  the  Air  Force  regularly  collects  this  type  of 
information. 

Job  Analysis  Data 

The  Air  Forces  uses  the  term  OA  to  describe  the  act  of  collecting  information  about  the  activi¬ 
ties  and  tasks  Air  Force  personnel  do  on  their  jobs.  Other  names  for  the  same  or  similar  data- 
collection  efforts  include  JA,  work  analysis,  or  task  analysis.  In  this  report,  we  use  JA  to  refer  to 
all  concepts,  except  when  we  are  speaking  specihcally  of  the  Air  Force’s  practices,  where  we 
use  OA.  At  its  most  general  level,  JA  is  a  systematic  investigation  into  the  work  that  people 
do  and  is,  thus,  one  of  the  most  fundamental  types  of  data  for  HRM.  Ash  (1988)  defines  it  as 
“attempts  to  reduce  to  words  the  things  people  do  in  human  work”  (p.  3).  Typical  questions  on 
a  JA  survey  deal  with  the  importance  or  frequency  of  job  tasks  (Williams  and  Crafts,  1997). 
In  other  cases,  data  collection  may  involve  observations  of  the  work  being  performed  or  inter¬ 
views  and  focus  groups  with  job  incumbents  (Schmitt  and  Chan,  1998).  Worker  characteristic 
requirements,  or  the  KSAOs  needed  for  the  job,  can  also  be  collected  or  identified  through  JA 
and  can  be  quite  useful  for  training  and  selection  purposes  (Brannick,  Levine,  and  Morgeson, 
2007).^ 

Table  2.2  highlights  some  example  research  questions  for  which  JA,  attitudinal,  and 
administrative  data  are  relevant.  As  noted  by  R  Wright  and  McMahan  (1992),  research  can 
focus  on  specific  HRM  practices  and  highlight  the  optimal  course  in  the  given  microfunc¬ 
tional  view.  However,  as  argued  by  Wright  and  McMahan  themselves  and  others  (e.g.,  Bowen 
and  Ostroff,  2004;  Huselid,  1995;  Lepak  et  ah,  2006),  the  appropriate  approach  is  a  systemic 
one.  Individual  personnel  practices  that  make  up  the  personnel  system,  or  the  development  of 
such  a  system,  should  rest  on  empirical  data.  The  data  enable  informed  decisions  about  how 
things  are  integrated  (or  not);  mismatches  provide  insight  into  constructing  policies  so  organi¬ 
zations  can  send  employees  consistent,  well-integrated  messages  about  needed  behaviors. 


^  Harvey  (1991)  notes  that  knowledge  and  skills  are  typically  more  easily  observable  through  behavior,  while  abilities  and 
“other”  characteristics  (such  as  personality  attributes)  are  less  directly  observable  on  the  job  but  may  be  assessed  through 
other  instruments  that  measure  the  constructs  in  question  (e.g.,  aptitude  or  personality  tests). 
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Table  2.2 

Examples  of  Research  Questions  for  Which  Job  Analysis,  Attitudinal,  or  Administrative  Data  Are 
Relevant 


Topic  Area 

Research  Question 

Attitude 

JA 

Administrative 

Recruiting 

How  do  you  generate  interest  among 
underrepresented  groups? 

Yes 

Yes 

Which  screening  tools  discourage  people 
from  applying  for  this  job? 

Yes 

Selection 

What  types  of  screening  tools  are  needed  for 
this  job? 

Yes 

Yes 

Are  perceptions  of  the  screening  process 
more  favorable  when  their  relationship  to  job 
requirements  is  explained? 

Yes 

Yes 

Training  and  education 

What  type  of  training  is  needed? 

Yes 

Do  trainees  plan  to  apply  the  training  on  the 
job?  Why  or  why  not? 

Yes 

Classification 

Which  jobs  are  similar? 

Yes 

Does  interest  matter  in  person-job  matching? 

Yes 

Yes 

Yes 

Monitoring  workplace 

Does  organizational  commitment  predict 

Yes 

Yes 

attitudes  and 

retention  decisions  or  actual  retention? 

perceptions 

Do  perceptions  of  base  services  affect 
performance? 

Yes 

Yes 

Yes 

Performance  evaluation 

Does  perceived  fairness  of  promotions  affect 
attendance? 

Yes 

Yes 

Is  the  promotion  process  identifying  the  best 
officers? 

Yes 

Yes 

There  are  various  ways  to  find  out  about  employee  work  lives  and  elicit  empirical  data  on 
how  employees  behave  on  the  job,  what  attitudes  motivate  these  behaviors,  and,  ultimately, 
how  these  attitudes  affect  performance  at  higher  levels  of  aggregation  (i.e.,  performance  at  the 
organizational,  rather  than  individual,  level).  In  the  next  section,  we  provide  more  detail  on 
the  role  of  job  data  in  each  personnel  function  listed  in  the  table  to  illustrate  the  importance 
of  various  HRM  practices  and  highlight  some  specific  applications  in  each  that  may  be  taken 
individually  as  “best  practices”  or,  more  optimally,  integrated  together  with  the  organization’s 
strategic  objective  in  mind.  Moreover,  by  describing  some  of  the  findings  and  applications  of 
personnel  research  over  the  years,  we  illuminate  its  usefulness  and  relevance  for  setting  person¬ 
nel  policy. 


Personnel  Research  Areas  Requiring  the  Three  Types  of  Data 

Recruiting 

Recruiting  is  the  preliminary  encounter  potential  employees  have  with  a  potential  employer 
and  the  prequel  of  the  selection  process.  For  the  Air  Force,  recruiting  appropriate  personnel  in 
sufficiently  large  numbers  is  the  starting  point  for  force  management  efforts.  The  restriction  on 
lateral  hiring  in  the  military  for  virtually  all  occupations  means  that  the  services  recruit  young. 
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inexperienced  individuals  and  attempt  to  nurture  them  through  their  careers,  with  sufEcient 
numbers  of  sufficient  quality  attaining  seniority  (i.e.,  promotion  to  higher  rank)  that  the  health 
of  the  service  is  ensured  (Asch  and  Warner,  2001).  Certain  aspects  of  the  recruitment  process 
require  information  about  the  jobs  themselves,  preferably  based  on  empirical  data,  including 
JA.  These  include  realistic  job  previews,  job  descriptions,  and  person-organization  ht  (P-O  ht). 

Realistic  job  previews  and  job  descriptions  are  exactly  what  they  imply:  presentations  of 
accurate  information  about  the  job  itself  and  a  description  of  the  work  involved.  Job  descrip¬ 
tions  may  be  used  to  attract  appropriate  applicants  to  the  selection  process  and  may  save  money 
by  preemptively  eliminating  individuals  who  would  not  enjoy  the  job  and  who  would  ulti¬ 
mately  attrit.  Realistic  job  previews  function  similarly,  although  the  effects  of  these  types  of 
selection  interventions  are  often  studied  over  a  longer  term.  Research  indicates  that  realistic  job 
previews  have  small  but  stable  effects  on  several  important  organizational  outcomes,  including 
reduction  in  voluntary  attrition  and  increased  performance  (Phillips,  1998). 

P-O  ht  (or  person-job  ht)  concerns  how  much  the  individual  and  the  organization  or  job 
suit  each  other.  Attitudinal  data  can  be  relevant  for  understanding  needs-supplies  ht,  one  type 
of  ht  that  describes  the  ways  in  which  employees’  needs  or  preferences  are  in  some  way  met  by 
their  jobs.  For  example.  Cable  and  DeRue  (2002)  demonstrated  that  perceptions  of  an  organi¬ 
zation’s  values  are  related  to  the  employees’  identihcation  with  the  organization  and  turnover 
decisions  (see  also  Kristof-Brown,  Zimmerman,  and  Johnson,  2005).  These  outcomes  are  key 
for  the  military  services  because  the  military  depends  on  attracting  quality  candidates  early  on 
and  retaining  these  employees  in  a  closed  labor  market. 

Selection 

Selection  identihes  those  applicants  with  the  KSAOs  necessary  to  perform  in  a  given  job.  JA, 
the  standard  method  in  determining  which  KSAOs  are  needed  in  a  given  job,  is  recognized  by 
both  the  courts  and  professional  practice  as  the  hallmark  of  well-supported  selection  systems. 
Although  the  legislation,  which  includes  the  Americans  with  Disabilities  Act  (Pub.  L.  101-336, 
1990)  and  the  Equal  Pay  Act  (Pub.  L.  88-38,  1963),  does  not  cite  JA  by  name,  the  Uniform 
Guidelines  on  Employee  Selection  Procedures  of  the  Equal  Employment  Opportunity  Com¬ 
mission  (EEOC)  (EEOC,  1978)  do  state  that  JA  should  be  the  basis  of  selection. 

The  essential  issue  these  regulations  and  laws  address  is  that  the  content  of  the  selection 
test  must  ideally  be  related  with  job  performance,  both  in  terms  of  apparent  content  and  in 
terms  of  actual  correlation:  JA  provides  information  about  what  tasks  are  performed  and  what 
skills  and  abilities  are  necessary  for  performance.  It  is  legal  to  reject  job  applicants  based  on 
job-related  requirements;  however,  legal  liability  may  be  incurred  if  the  reason  for  rejection  is 
deemed  unfair  and  not  job-related.  Although  the  military  is  not  subject  to  some  of  the  legal 
requirements  that  dictate  employment  in  the  civilian  sector,  it  typically  follows  best  practices 
and,  where  it  deems  practicable  given  the  constraints  of  the  military  mission,  best  practices  in 
selection. 

Training  and  Education 

Brannick,  Eevine,  and  Morgeson  (2007)  note  that  almost  all  jobs  require  some  training.  To  the 
extent  that  a  given  job  requires  an  employee  to  do  unfamiliar  tasks,  training  may  be  required, 
especially  if  the  costs  of  mistakes  are  high.  The  hrst  step  in  developing  a  good  training  program 
is  identifying  the  gap  between  what  is  needed  and  what  is  available  (Campbell  and  Kuncel, 
2002).  For  entry-level  employees,  hlling  this  gap  should  be  included  in  their  entry-level  train- 
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ing.  Christal  and  Weissmuller  (1988)  indicate  that  the  military  and  the  Air  Force  specifically 
have  used  JA  (or  OA)  to  inform  training  decisions  with  notable  success,  saving  millions  of 
dollars  over  the  years.  For  example,  training  is  used  to  ready  entry-level  employees  for  techni¬ 
cal  specialties,  and  OA  has  proven  quite  helpful  in  determining  task  training  priority  (Ruck, 
Thompson,  and  Stacy,  1987). 

Employee  learning  happens  over  a  career  (McCauley  and  Hezlett,  2002),  particularly  in 
the  Air  Force,  where  career  paths  involve  multiple  jobs,  sometimes  in  other  Air  Force  special¬ 
ties  (AFSs)  for  which  the  requisite  training  may  differ  (e.g.,  Schiefer  et  ah,  2007).  JA  data  can 
be  used  to  assess  commonality  of  tasks  across  jobs  and  help  estimate  the  resources  required 
for  retraining  (Lance,  Kavanagh,  and  Gould,  1993;  Lance,  Mayfield,  and  Gould,  1993). 
J.  Mitchell,  Yadrick,  and  Bennett  (1996)  describe  some  of  the  issues  in  developing  a  truly  inte¬ 
grated  training  system,  including  conceptualizing  career  paths,  inclusive  of  learning  assign¬ 
ments  and  opportunities  along  the  way,  and  identifying  when  retraining  will  be  necessary. 

Merely  receiving  training  is  not  enough;  that  training  must  be  applied,  or  transferred, 
from  the  training  environment  to  the  organizational  and  operational  environment.  L.  Burke 
and  Hutchins  (2007)  reviewed  the  literature  on  this  topic  and  found  that  estimates  of  amount 
of  training  transfer  varied  widely  from  as  little  as  10  percent  to  50  percent — that  is,  of  train¬ 
ing  received,  only  about  half  of  it,  at  most,  is  put  into  action.  Burke  and  Hutchins’  summary 
indicated  that  climate  factors — including  peer  and  supervisory  support  for  transfer,  transfer 
climate  factors,  and  opportunity  to  perform  trained  behaviors — were  on  the  whole  a  strong 
predictor  of  training  success:  Gonsideration  of  these  issues  enables  training  to  pay  off.  Lord  et 
al.  (1992)  demonstrated  this  specifically  in  the  Air  Lorce  environment;  they  showed  that  super¬ 
visor  attitudes  and  coworker  support  were  key  predictors  of  the  breadth  of  opportunity  for 
task  performance  on  the  job  and  of  the  complexity  of  the  trained  tasks  themselves.  Therefore, 
several  different  types  of  personnel  data  can  be  useful  in  improving  training,  ranging  from  JA 
to  data  collected  on  attitudes. 

Classification 

Glassifying  jobs  involves  determining  the  similarity  of  different  jobs  for  a  given  personnel 
purpose,  such  as  selection,  training,  or  compensation,  and  JA  data  are  integral  for  this  pur¬ 
pose  (Harvey,  1991).  One  of  the  key  purposes  of  OA  data  in  the  Air  Lorce  has  been  providing 
guidance  for  grouping  and  ungrouping  ALSs  (Ghristal  and  Weissmuller,  1988).  Lor  exam¬ 
ple,  Driskill  et  al.  (1989)  explored  the  utility  of  various  JA  techniques  for  suggesting  abilities 
required  for  Air  Lorce  jobs  (i.e.,  facilitating  selection  and  classification);  Earles,  Driskill,  and 
Dittmar  (1996)  discussed  another  approach  using  subject-matter  expert  (SME)  ratings  to  make 
the  inferential  leap  between  task  statements  and  ability  requirements. 

Grouping  jobs  based  either  on  tasks  in  common  or  the  abilities  required  to  perform  the 
tasks  helps  in  reassigning  personnel.  That  is,  should  an  employee  need  to  change  jobs  for  some 
reason,  information  about  the  employee’s  current  tasks  and  the  skills  and  abilities  used  to  do 
those  tasks,  as  well  as  the  tasks  performed  and  abilities  needed  for  available  jobs  more  generally, 
will  enable  that  employee  to  be  reassigned  to  another  job  that  most  closely  matches  his  or  her 
existing  tasks  and  abilities  and  hence  presents  the  least  expensive  option  for  the  organization 
in  terms  of  retraining. 

Linally,  classification  serves  another  purpose  related  to  selection.  Once  the  abilities  needed 
for  a  job  are  known  and  some  minimum  requirements  met,  an  employee  may  be  selected  into 
a  particular  job  based  on  the  abilities  he  or  she  has — what  is  called  classification  of  personnel 
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(rather  than  classification  of  jobs).  Again,  deciding  how  to  classify  personnel  requires  knowl¬ 
edge  about  the  jobs  themselves  (and,  hence,  JA  data).  Other  decisions  in  this  process  include 
what  outcomes  to  maximize  through  this  process  and  potentially  require  other  types  of  data: 
aggregate  job  performance  across  all  assignments,  social  benefits  (e.g.,  percentage  of  minority 
placements),  or  job  and  career  satisfaction.  (See  Rosse,  Campbell,  and  Peterson,  2001,  for  an 
in-depth  discussion  of  these  issues.) 

Monitoring  Workplace  Attitudes  and  Perceptions 

Monitoring  work  attitudes  and  perceptions  is  not  a  “classic”  personnel  research  function  in 
that  attitudes  and  perceptions  were  less  a  focus  of  personnel  research  in  its  early  days;  how¬ 
ever,  in  recent  decades,  their  importance  has  led  to  increasing  attention.  Work  attitudes  and 
perceptions  include  such  factors  as  engagement,  fairness  perceptions,  organizational  commit¬ 
ment,  and  job  satisfaction  (overall  satisfaction  or  satisfaction  with  individual  facets,  such  as 
supervision,  coworkers,  the  work  itself,  pay,  benefits,  or  promotions).  Employee  perceptions  of 
organizational  climate  and  culture  can  also  fall  under  this  broad  umbrella,  including  diversity 
climate,  safety  climate,  climate  for  training  transfer,  and  perceptions  of  organizational  values. 
We  focus  on  job  satisfaction  and  safety  climate  here  for  illustrative  purposes.  Many  of  the  same 
general  findings  are  similar  for  other  workplace  attitudes  and  perceptions. 

Roznowski  and  Hulin  (1992)  argue  that  a  good  measure  of  job  satisfaction  provides  an 
organization  with  the  single  best  way  to  predict  its  employees’  behavior.  Job  satisfaction  is  one 
of  the  best-known  and  most-studied  causes  of  organizational  attrition  (Horn  and  Griffeth, 
1995)  and  has  been  linked  with  performance  at  the  individual  (Judge  et  ah,  2001)  and  unit 
levels  (Harter  and  Schmidt,  2006).  In  a  military  context  specifically,  Jordan  et  al.  (2007)  found 
that  job  satisfaction  predicted  organizational  citizenship  behavior  among  officers  (helpful  work 
activities  that  go  beyond  what  is  required  on  the  job,  such  as  assisting  a  colleague  to  complete 
an  important  task  when  that  is  not  part  of  one’s  regular  duties)  enrolled  in  a  professional  mili¬ 
tary  education  (PME)  course  (their  participants  were  primarily  Air  Force  officers).  Saari  and 
Judge  (2004)  note  that  making  work  interesting  and  challenging  for  employees  may  be  more 
effective  in  increasing  job  satisfaction  than  pay  interventions.  This  also  suggests  that  it  is  help¬ 
ful  to  integrate  information  about  the  work  situation  (perhaps  obtained  through  JA)  with  job 
satisfaction  information  to  modify  jobs  to  best  retain  employees  and  motivate  them  to  higher 
levels  of  performance. 

Given  that  safety  climate  can  affect  safety  behaviors  at  work  (Glarke,  2006),  organiza¬ 
tions  also  often  aim  to  measure  and  improve  it.  Climate  is  generally  defined  as  the  aggregated 
employee  perceptions  of  the  workplace’s  policies,  practices,  and  procedures  about  a  specific 
content  area.  In  this  sense,  safety  climate  could  be  defined  as  perceptions  of  management  atti¬ 
tudes  toward  safety,  effects  of  safety  on  promotions  and  social  status  outcomes,  importance 
and  effectiveness  of  safety  training  and  enforcement  guidance,  workplace  risk  level,  and  the 
status  of  the  safety  officer  (definition  from  Zohar,  1980). ^  Once  perceptions  of  the  different 
facets  of  safety  climate  (e.g.,  perceptions  of  management’s  attitudes  toward  safety)  are  known, 
an  organization  can  attempt  to  change  those  perceptions  through  various  interventions;  for 
example,  leadership  training  interventions  have  been  developed  and  shown  to  be  successful 
(Zohar,  2002). 


^  Clarke  (2006)  notes  that  this  definition  still  has  currency  today,  although  other,  broader  conceptualizations  have  been 
suggested. 
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The  above  examples  illustrate  the  important  role  that  monitoring  workplace  attitudes  and 
perceptions  can  have  on  employee  and,  ultimately,  organizational  effectiveness.  But  JA  data 
can  be  useful  here,  too.  For  example,  if  making  work  interesting  and  challenging  is  impor¬ 
tant  for  job  satisfaction,  it  would  be  helpful  to  have  a  picture  of  the  tasks  employees  do  on  the 
job.  Similarly,  to  evaluate  safety  climate,  one  should  know  what  types  of  safety  activities  are 
required  on  the  job. 

Performance  Evaluations 

Ultimately,  organizations  hire  employees  to  do  a  job,  and  how  well  employees  do  their  job,  or 
their  performance,  is  assessed  throughout  their  time  with  the  organization.  These  evaluations  of 
performance  can  occur  for  a  variety  of  different  purposes,  including  providing  developmental 
feedback;  identifying  whom  to  promote;  identifying  those  deserving  of  bonuses,  awards,  and 
recognition;  diagnosing  training  needs;  allocating  annual  pay  raises;  and  identifying  employ¬ 
ees  whose  behavior  needs  corrective  actions.  As  with  selection,  basing  performance  evaluation 
on  a  careful  JA  (e.g.,  potentially  OA  data)  is  an  important  defense  against  legal  challenges 
(Brannick,  Levine,  and  Morgeson,  2007;  Latham  and  Mann,  2006);  and  performance  man¬ 
agement  is  the  most  common  ground  for  legal  challenge  (Latham  and  Mann,  2006;  Newman, 
Kinney,  and  Farr,  2004).  Basing  performance  rating  scales  on  JA  data  is  also  consistent  with 
good  professional  practice  in  developing  performance  evaluation  systems  (Brannick,  Levine, 
and  Morgeson,  2007).*’ 

To  illustrate  how  tying  performance  evaluation  to  JA  data  can  be  important,  we  briefly 
describe  its  relevance  in  one  type  of  performance  evaluation:  360-degree  feedback.  Quality 
feedback  allows  employees  to  identify  areas  in  need  of  improvement  and  to  set  specific  dif¬ 
ficult  but  attainable  goals  for  improving  those  areas  (Latham  and  Mann,  2006;  DeNisi  and 
Kluger,  2000);  feedback  perceived  as  fair  is  also  more  likely  to  be  accepted  (see,  e.g.,  Flint, 
1999).  Three-hundred-sixty-degree  feedback,  which  uses  data  from  people  all  around  the  focal 
employee  (e.g.,  peers,  subordinates,  customers,  clients,  and  mentors,  as  well  as  superiors),  is  use¬ 
less  if  the  feedback  is  not  based  on  actual  job  requirements  or  is  vague  or  abstract.  Creating  a 
feedback  rating  system  that  forces  raters  to  evaluate  a  person’s  performance  on  each  aspect  of 
the  job  helps  ensure  that  the  feedback  both  is  interpretable  and  accurately  accounts  for  the  per¬ 
son’s  behavior  on  the  job.  In  this  way,  critical  aspects  of  the  job  cannot  be  accidentally  left  out, 
and  irrelevant  aspects  of  the  job  are  less  likely  to  influence  the  ratings.  Moreover,  the  feedback 
may  be  more  likely  to  be  viewed  as  fair  and  to  be  integrated  into  actual  performance  (Flint, 
1999).  Personnel  research  and  data  collection  are  core  parts  of  designing  a  good  360-degree 
feedback  program,  just  as  they  are  vital  components  of  all  types  of  performance  evaluations. 


Longitudinal  Research 

Much  personnel  data  are  collected  longitudinally.  Administrative  data  track  organizational 
start  dates,  pay  periods,  and  pay  rates  over  time,  for  tax  purposes  if  for  nothing  else.  However, 


®  Stone  et  al.  (1996)  provide  an  example  of  directly  applying  OA  knowledge  to  performance  management  in  the  Air  Force; 
they  used  information  about  the  number  of  core  tasks  (those  performed  by  a  high  percentage  of  airmen  in  the  same  jobs,  at 
the  same  level)  performed,  the  number  of  noncore  tasks  performed,  and  the  difficulty  of  those  tasks  to  develop  a  system  for 
rating  job  performance. 
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this  represents  the  hare  minimum  of  personnel  data  that  can  he  collected  longitudinally.  An 
employee’s  experience  of  the  organization  is  inherently  temporal,  progressing  from  recruitment 
to  selection  to  some  combination  of  training,  performance  management,  and  performance, 
until  the  point  of  employee  departure  or  turnover.  In  an  ideal  organization,  personnel  data  are 
collected  throughout  to  monitor,  predict,  and  enhance  this  experience,  both  for  the  employee 
and  for  the  organization. 

When  using  personnel  research  to  capture  employees’  experience  on  the  job  and  predict 
future  behaviors,  George  and  Jones  (2000)  point  out  that  a  consideration  of  time  lags  between 
data-collection  points,  anticipated  duration  of  the  effects  being  examined,  and  temporal  aggre¬ 
gation  help  articulate  the  true  nature  of  organizational  experiences  and  the  causal  relationships 
of  interest.  Data  collection  that  represents  a  single  snapshot  in  time  does  not  adequately  portray 
organizational  realities,  given  the  inherent  temporality  of  experience.  In  addition,  some  impor¬ 
tant  organizational  outcomes,  such  as  absenteeism  and  turnover,  tend  to  occur  so  infrequently 
that  studying  those  behaviors  in  isolation  requires  aggregation  of  incidences  of  absenteeism  and 
turnover  over  time  (e.g.,  Hulin,  1991). 

Adequate  time  lags  are  also  required  to  determine  whether  various  personnel  practices  are, 
in  fact,  bringing  about  desired  change.  Consider  the  training  effectiveness  criteria  of  behavior 
on  the  job:  Inherent  in  any  evaluation  of  this  is  sufficient  time  after  the  training  occurs  for 
trainees  to  demonstrate  behavior  on  the  job.  Further,  George  and  Jones  (2000)  note  that,  if 
theory  or  empirical  evidence  suggests  that  the  effects  of  job  redesign  have  a  lagged  effect  on 
performance,  performance  data  should  be  collected  when  the  impact  is  expected  to  manifest 
or  risk  drawing  the  erroneous  conclusion  that  the  intervention  was  not  successful,  as  well  as  a 
lost  opportunity  to  utilize  a  validated  and  successful  intervention  in  the  future. 

Longitudinal  data  collection  is  thus  essential  to  ensure  the  proper  evaluation  of  key  per¬ 
sonnel  functions  and  the  effectiveness  of  organizational  interventions.  Although  W.  Warner 
Burke  (2006)  notes  that  surveys  may  be  the  most  useful  method  for  tracking  change  over  an 
organizational  intervention,  any  data  collection  is  an  opportunity.  Indeed,  some  longitudinal 
research  combines  administrative  data,  such  as  turnover  records,  with  other  types  of  data,  such 
as  job  attitude  data  obtained  from  surveys.  All  that  is  required  is  organization  and  tracking  of 
individuals  over  time.  Proper  considerations  for  the  security  and  privacy  of  identifiable  data 
are,  of  course,  an  essential  requirement  here;  moreover,  institution  of  tracking — or  perceptions 
of  tracking — when  none  had  previously  existed  should  itself  be  examined  as  a  potential  bias¬ 
ing  factor  in  the  results  obtained.  In  an  ideal  organization,  longitudinal  data  collection  and 
research  may  also  capitalize  on  the  ongoing  collection  of  personnel  data.  If  an  organization 
regularly  assesses  climate  and  job  attitudes,  for  example,  collecting  information  that  allows 
these  personnel  data  to  be  linked  with  administrative  data,  such  as  job  changes,  training,  and 
performance  management  efforts,  allows  the  organization  to  determine  the  effect  of  these 
organizational  experiences  without  needing  further  data  collection  to  elicit  self-reported  job 
changes,  training,  and  performance.  (Note,  however,  that  self-reported  data  may  also  be  sub¬ 
ject  to  various  biases,  including  those  precipitated  by  perceptions  of  confidentiality  and  ano¬ 
nymity  of  the  data,  or  the  lack  thereof.^ 

Several  authors  caution  against  surveying  employees  merely  for  the  sake  of  obtaining 
information  (e.g.,  Donovan  and  Brooks,  2006;  Ghurch  and  Oliver,  2006;  Smith,  2003).  Lon- 


^  Some  research  has  explored  these  issues.  See,  e.g.,  Joinson,  Woodley,  and  Reips  (2007);  Rogelberg  et  al.  (2006);  and 
Spitzmiiller  and  Glenn  (2006)  for  discussion  of  nonresponse,  confidentiality,  and  anonymity. 
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gitudinal  research  and  personnel  data  collection  protect  against  repetitive  oversurveying  by 
allowing  one  data  collection  to  be  used  for  multiple  purposes;  they  can  also  help  show  employ¬ 
ees  the  impact  and  application  of  the  data  collected  in  terms  of  the  cycle  of  organizational 
growth,  change,  development,  and  interventions.  Donovan  and  Brooks,  among  others,  empha¬ 
size  that  merely  providing  feedback  reports  to  employees  is  inadequate:  For  employees  to  see 
these  data  collections  as  having  merit  and  being  worth  their  time  and  effort,  the  organization 
must  be  seen  as  acting  on  the  information  collected,  whatever  its  form. 


Summary 

Recruitment,  selection,  classification,  training,  performance  management,  and  work  attitudes 
interact  throughout  an  employee’s  tenure  on  the  job.  Recruitment  should  focus  on  individuals 
with  the  best  fit  between  person  and  organization,  both  in  terms  of  values  and  KSAOs,  and 
selection  should  further  narrow  the  field  to  allow  entry  only  to  those  employees  with  the  best 
potential  to  facilitate  the  organization’s  goals.  Classification  of  personnel  continues  this  course, 
matching  employees  with  jobs  in  which  they  are  most  likely  to  thrive.  Training,  if  needed, 
further  prepares  the  employees  with  needed  bodies  of  knowledge  and  skill  sets.  Classification 
of  jobs  indicates  where  job  similarities  will  help  employees  transfer  successfully  between  jobs 
and  which  jobs  may  be  combined  or  separated  most  efficiently.  Performance  management  over 
an  employee’s  tenure  facilitates  cohesive  employee  development  and  provides  a  way  to  reinforce 
desired  behaviors.  Attention  to  work  attitudes  throughout  can  facilitate  transfer  of  training, 
performance  of  valued  behaviors,  and  organizational  change  efforts. 

These  HRM  practices  are  best  informed  by  well-executed  data — ^JA  data,  attitudinal 
data,  and  administrative  data — gathered  through  surveys,  interviews,  focus  groups,  observa¬ 
tion,  and  human  resource  recordkeeping.  Ideally,  these  data  are  recorded  regularly  and  longi¬ 
tudinally  to  better  model  organizational  experience  and  change.  Aligning  the  HRM  system 
as  a  whole  facilitates  the  attainment  of  strategic  goals  and  enables  an  organization  to  establish 
a  strong,  consistent  climate.  Without  information  about  employee  experiences  on  the  job,  an 
organization  may  blindly  grope  its  way  along  and  bypass  the  advantage  of  data-driven  HRM 
decisions. 


CHAPTER  THREE 


A  Very  Brief  History  of  the  Management  of  Personnel  Research  in 
the  Air  Force 


Personnel  research  in  the  Air  Force  has  its  historical  roots  in  two  organizations:  AFHRL  and 
AFOMSd  Across  its  lifetime,  AFHRL  produced  a  wealth  of  personnel  research  on  topics  span¬ 
ning  all  types  of  Air  Force  personnel  issues.  A  product  of  AFHRL  research,  AFOMS  was  an 
organization  dedicated  to  regularly  collecting  and  applying  OA  (i.e.,  JA)  data  in  the  Air  Force. 
This  chapter  provides  a  brief  history  of  both  organizations  to  describe  the  Air  Force’s  former 
personnel  research  and  development  of  strategic  alignment  of  that  research  as  a  preface  to  a 
discussion  of  the  current  state  of  personnel  research  in  the  Air  Force.^  Note  that,  although 
we  are  discussing  programmatic  research  efforts,  individual  studies  were  historically  and  are 
now  conducted  in  the  Air  Force.  Unfortunately,  in  contrast  to  the  historical  roots  of  personnel 
research  in  the  Air  Force,  programmatic  and  strategic  personnel  research  is  all  but  precluded 
in  the  present. 


The  Air  Force  Human  Resources  Laboratory 

Following  World  War  II,  the  success  of  the  aviation  psychology  program  that  helped  identify 
potential  airmen  spurred  an  interest  in  personnel  research  and  management  (Brokaw  and  Per- 
rigo,  1981).  This,  in  turn,  led  to  the  establishment  of  several  personnel  research  organizations 
from  1949  onward  that  focused  on  a  variety  of  issues,  including  training,  selection,  classifica¬ 
tion,  and  manpower  planning.  AFHRL  was  established  in  1968  after  a  special  panel  of  the 
Air  Force  Scientific  Advisory  Board  reviewed  the  Air  Force’s  behavioral  science  program  and 
recommended  that  the  program  be  expanded  by  creating  a  new  oversight  organization  and 
adding  staff  and  funding.  As  a  result,  AFHRL  was  established  as  an  Air  Force  Systems  Com¬ 
mand  (AFSC)  laboratory  and  designed  to  be  a  centralized  organization  supporting  manpower, 
personnel,  and  training  research  and  development.  Along  with  its  headquarters  at  Brooks  Air 
Force  Base,  Texas,  AFHRL  included  field  programs  collocated  within  other  operating  orga¬ 
nizations  and  bases.  With  its  establishment,  AFHRL  also  incorporated  the  already  existing 
Personnel  Research  Laboratory  and  the  Training  Research  Division  of  the  Aerospace  Medi¬ 
cal  Research  Laboratory  under  its  umbrella.  Overall,  AFHRL  was  responsible  for  “planning 
and  executing  the  USAF  [U.S.  Air  Force]  exploratory  and  advanced  development  programs 
for  selection,  motivation,  training,  retention,  education,  utilization  and  career  development 


'  Note  that  these  organizations’  names  varied  over  the  years. 

^  Sources  used  in  this  chapter  include  reports  that  are  not  generally  available  and  materials  we  were  given  during  inter¬ 
views.  Where  possible,  we  have  cited  relevant  and  generally  available  documentation. 
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of  military  personnel;  also  the  composition  of  the  personnel  force  and  training  equipment” 
(AFHRL,  1972). 

Originally,  AFHRL  consisted  of  two  divisions:  the  Advanced  Systems  Division  and  the 
Personnel  Research  Division.  By  the  end  of  1972,  AFHRL  had  expanded  to  five  divisions 
located  across  the  country: 

•  Flying  Training  Division 

•  Manpower  Development  Division 

•  Personnel  Research  Division  (Occupational  and  Career  Development  Branch,  Personnel 

Systems  Branch,  Computer  and  Management  Sciences  Branch) 

•  Advanced  Systems  Division  (Training  Technology  Branch,  Personnel  and  Training 

Requirements  Branch,  Simulation  Techniques  Branch,  Resource  Instrumentation  Branch) 

•  Technical  Training  Division. 

By  1972,  AFHRL  also  had  365  authorized  manpower  positions.  Its  staff  was  composed 
predominantly  of  civilians  (63  percent),  followed  hy  airmen  (23  percent),  and  officers  (14  per¬ 
cent).  Thirteen  percent  of  personnel  had  Ph.D.’s,  27  percent  had  master’s  degrees,  and  19  per¬ 
cent  had  bachelor’s  degrees,  while  41  percent  did  not  hold  a  college  degree.  AFHRL  personnel 
also  had  a  wide  variety  of  expertise,  including  psychologists,  economists,  operations  research¬ 
ers,  engineers,  and  mathematicians.  The  total  budget  at  the  time  was  more  than  $12  million 
(more  than  $64  million  in  2011  dollars). 

In  1983,  AFHRL  was  assigned  to  the  Aerospace  Medical  Division.  Then,  in  1991,  AFHRL 
was  incorporated  into  the  newly  established  Armstrong  Laboratory  as  the  Human  Resources 
Directorate  (AL/HR).  Like  the  original  AFHRL,  this  new  directorate  was  responsible  for 
“planning  and  executing  the  Air  Force  exploratory  and  advanced  development  programs  for 
research  and  development  (R&D)  related  to  manpower  and  force  management,  logistics  sys¬ 
tems  technology,  and  training  technology”  (Buescher,  Olvera,  and  Besetsny,  1989,  p.  2).  This 
included  a  manpower  and  personnel  program  in  selection,  classification,  retention,  force  struc¬ 
ture  and  force  utilization;  education  and  training  programs  in  technical  training,  flying  train¬ 
ing,  and  crew  and  team  training;  simulation  and  training  device  programs  to  develop  flight 
simulators  and  maintenance  training  simulators;  and  logistics  and  human  factors  programs  in 
weapon  system  logistics  and  combat  maintenance. 

In  1997,  the  Armstrong  Laboratory  was  merged  with  four  other  Air  Force  laboratories 
and  the  Air  Force  Office  of  Scientific  Research  to  form  AFRL,  which  was  subordinate  to  Air 
Force  Materiel  Command  (AFMC).  Following  this  merger,  AFRL  decided  in  1998  to  discon¬ 
tinue  funding  for  manpower  and  personnel  research  within  the  Human  Effectiveness  Direc¬ 
torate.  In  its  place,  funding  for  maintaining  longitudinal  databases  of  personnel  research  was 
to  be  continued,  with  contractors  filling  in  the  missing  expertise.  Unfortunately,  the  positions 
designated  to  maintain  the  personnel  research  databases  were  reassigned  to  meet  other  person¬ 
nel  priorities,  leaving  the  databases  unmaintained  for  several  years  (these  databases  are  being 
restored  by  AFPC’s  Strategic  Research  and  Assessments  Branch,  or  SRA).  As  a  result  of  this 
disestablishment,  the  work  by  AFHRL  and  its  later  heirs  was  discontinued  or  taken  up  by  other 
organizations  dispersed  across  the  Air  Force. ^  The  Air  Force  has  not  had  another  organization 


^  Some  of  the  original  facets  have  persisted  (i.e.,  training  research  has  retained  a  research  and  development  function  within 
AFRL/Human  Effectiveness). 
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solely  dedicated  to  research  and  development  for  personnel  purposes  on  this  hroad  of  a  scale 
since,  and  the  general-purpose  manpower  and  personnel  and  human  resources  research  func¬ 
tion  has  not  been  a  priority  for  many  years,  as  exemplified  hy  the  lack  of  resources  devoted  to 
it.  The  absence  of  a  single  organization  with  a  core  of  personnel  research  expertise  has  had  some 
notable  consequences,  including  the  lack  of  a  single  organization  to  serve  as  a  central  resource 
for  potential  consumers  of  personnel  research.  Other  consequences,  such  as  a  lack  of  in-house 
expertise  and  sometimes-limited  data  application  and  sharing,  may  be  only  secondarily  related 
to  the  decline  of  AFHRL  and  its  heirs  and  attributable  more  to  the  loss  of  coherent  organiza¬ 
tional  memory,  dispersion  of  endeavor,  and  overall  deprioritization  of  the  HRM  mission. 

Over  the  years,  AFHRL  and  its  heirs  researched  a  wide  range  of  personnel  topics.  Follow¬ 
ing  are  some  examples:^ 

•  developing  the  Weighted  Airman  Promotion  System  (WAPS) 

•  examining  the  officer  promotion  system 

•  developing  the  methodology  for  collecting,  analyzing,  and  reporting  Air  Force  OA 

•  evaluating  the  effectiveness  of  aircraft  simulations 

•  developing  formulas  for  Air  Force  Reserve  Officer  Training  Corps  (AFROTC)  pilot 
trainee  selection 

•  creating  pilot  performance  measures  for  undergraduate  pilot  training 

•  determining  aptitude  requirements  for  various  AFSs 

•  developing  tests,  including  the  Air  Force  Officer  Qualifying  Test  (AFOQT) 

•  evaluating  new  training  techniques 

•  evaluating  person-job  match 

•  measuring  the  effects  of  job  satisfaction  on  reenlistment  intentions 

•  determining  the  role  of  personality  in  job  performance. 


The  Air  Force  Occupational  Measurement  Squadron 

AFHRL  originally  designed  and  implemented  Air  Force  OA  and  the  WAPS.  However,  OA 
data  collection  and  WAPS  test  development  were  eventually  deemed  to  be  operational  func¬ 
tions;  thus,  in  1970,  the  occupational  testing  program  was  combined  with  the  occupational 
survey  program  to  form  Detachment  17  of  the  3300th  support  squadron  under  Air  Training 
Command.^  The  new  organization  consisted  of  staff  trained  in  implementing  the  OA  and 
WAPS  test  routines  designed  by  the  AFHRL  researchers.  In  1974,  Detachment  17  became  the 
Air  Force  Occupational  Measurement  Center,  and  later  AFOMS  (A.  Wright,  2009). 


^  For  annotated  bibliographies  of  AFHRL’s  research  from  1946  through  1995,  see  AFFIRL,  annotated  bibliographies 
1946-1995. 

5  The  Air  Force  sometimes  distinguishes  operational  activities  from  research  activities  based  on  whether  a  data-collection 
effort  is  carried  out  regularly  (e.g.,  annually)  as  part  of  a  routine  process.  Data  collection  that  is  not  regular  and  routine 
tends  to  be  viewed  as  research,  whereas  data  collection  that  is  routine  and  regular  is  considered  an  operational  data- 
collection  activity.  This  distinction  comes  from  official  guidance  regulating  funding  streams  for  research  (i.e..  Major  Force 
Program  [MFP]  6  is  devoted  to  research,  development,  test,  and  evaluation  funding,  while  other  streams,  such  as  MFP  8, 
are  devoted  to  more-routine  maintenance  efforts).  Note  also  that  this  distinction  is  not  one  we  use  in  our  definition  of  per¬ 
sonnel  research,  all  data-collection  activities,  whether  regular  and  routine  or  not,  are  considered  personnel  research  under 
our  definition. 
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AFOMS’s  primary  mission  was  collecting  OA  data  and  using  those  data  to  inform  the 
content  of  enlisted  training  programs  and  develop  Specialty  Knowledge  Tests  (SKTs)  for  the 
WAPS.  AETC  Instruction  (AETCI)  36-2601  (1999)  describes  AEOMS  as  the  “program  which 
collects  and  maintains  an  occupational  database  to  provide  information  about  Air  Eorce  jobs. 
The  process  involves  developing  a  survey  questionnaire,  collecting  and  analyzing  data,  and 
reporting  resulting  survey  information”  (p.  6).  AEOMS  routinely  provided  OA  data  to  Air 
Eorce  career  held  managers  (OEMs),  training  personnel,  and  major  command  (MAJCOM) 
functional  managers.  According  to  one  interviewee,  the  survey  data  were  “designed  to  facili¬ 
tate  preparation  for  utilization  and  training  workshops  (U&TW)  and  other  periodic  reviews 
of  personnel  classihcation,  training,  and  utilization  programs  and  practices.”  AEOMS  also 
developed  data  extracts  to  address  other  specihc  issues  as  needed  (AETCI  36-2601,  1999).  In 
1997,  AEOMS  became  an  EOA  reporting  to  the  Headquarters  AETC  Directorate  of  Opera¬ 
tions  (HQ  AETC/DO),  now  AETC  A2/A3/A10.  Originally,  the  squadron  consisted  of  four 
flights,  but,  as  a  result  of  downsizing,  only  two  flights  remain:  the  OA  flight  and  the  Test  and 
Evaluation  (TXTE)  flight.  Although  OA  and  TXTE  are  the  official  names  on  record  for  the 
two  flights,  the  flights  have  come  to  be  known  under  two  different  names:  OAD  and  AAD  for 
the  OA  and  TXTE  flights,  respectively.  These  are  the  names  they  used  when  referring  to  their 
organizations  in  our  interviews,  and  some  Air  Eorce  policy  documents  use  the  same  names 
when  making  reference  to  them.'"  Because  these  are  the  names  by  which  the  flights  have  self- 
identifled,  and  they  are  no  longer  referring  to  themselves  as  a  single  organization  known  as 
AFOMS,  we  have  adopted  the  two  names  they  provided  to  us  and  by  which  they  are  commonly 
known.  However,  as  AETC  Manpower  and  Personnel  (AETC/Al)  subject-matter  experts  have 
noted,  to  date,  they  are  not  officially  designated  as  divisions;  they  are  instead  technically  still 
considered  two  separate  flights  within  an  Air  Eorce- approved  squadron.  These  organizations 
and  their  continuing  data  collection  and  applications  are  discussed  further  in  the  next  chapter, 
on  current  Air  Eorce  personnel  research  and  data-collection  efforts. 


Summary 

AEHRE  and  its  heirs  were  historically  the  organizations  tasked  with  responsibility  for  all  per¬ 
sonnel  research.  Eor  many  years,  no  organization  has  had  that  same  official  level  of  responsi¬ 
bility.  Nevertheless,  OA  data  continue  to  be  collected  and  utilized  for  purposes  of  informing 
change  in  tech  training  and  designing  WAPS  tests,  and  other  organizations  accommodate 
requests  to  conduct  personnel  research  or  have  taken  on  responsibility  for  specific  aspects  of 
personnel  research.  These  organizations  and  their  personnel  research  efforts  are  described  in 
detail  in  the  next  chapter.  Although  some  research  and  the  collection  of  OA  data  have  contin¬ 
ued,  since  the  last  broadly  focused  personnel  research  vestige  of  AEHRE  at  AERE  was  elimi¬ 
nated,  there  has  been  no  single  resource  for  consumers  of  personnel  research  and  development, 
and  there  is  no  obvious  organization  for  consumers  (including,  of  course,  those  who  set  person¬ 
nel  policy)  to  which  to  turn  for  help  and  strategic  guidance. 
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See,  for  example,  Air  Force  Instruction  (AFI)  36-2623,  2012,  and  Air  Force  Pamphlet  (AFPAM)  36-2241,  2013. 


CHAPTER  FOUR 


Personnel  Research  and  Data  Collection  in  the  Air  Force  as  It 
Existed  in  2011 


As  noted  in  the  previous  chapter,  AFHRL  was  the  Air  Force  organization  historically  tasked 
with  responsibility  for  all  personnel  research  and  development.  With  AFHRL  and  its  heirs 
gone  as  the  centralized  source  of  such  research,  no  organization  now  has  that  same  official  level 
of  responsihility.  But  that  does  not  mean  that  personnel  research  is  not  being  done  now;  rather, 
it  is  being  done,  but  in  a  more  decentralized  and  less  strategic  way.  Today,  personnel  research 
is  taking  place  in  a  wide  variety  of  organizations,  with  no  single  organization  clearly  in  charge, 
and  none  with  a  mission  set  that  includes  strategic  research  and  development  for  the  personnel 
system  as  a  whole. 

In  this  chapter,  we  provide  an  overview  of  those  organizations — when  we  presented  our 
hndings  in  2011 — to  illustrate  the  decentralized  nature  of  the  current  personnel  research  in  the 
Air  Force  and  highlight  various  issues  associated  with  this  current  decentralized  structure  of 
personnel  research.  Specifically,  for  each  identified  organization,  we  discuss  its  position  within 
the  Air  Force  organizational  structure,  summarize  its  overall  mission,  describe  the  suborgani¬ 
zations  involved  in  personnel  research,  and  discuss  the  following  points  about  their  personnel 
research  efforts:  the  types  of  personnel  data  they  collect,  the  data-collection  methods  they  use, 
and  whatever  primary  and  secondary  uses  they  make  of  the  data.' 


What  Air  Force  Organizations  Now  Conduct  Personnel  Research? 

We  took  a  multistep  approach  to  identify  Air  Force  organizations  now  engaged  in  personnel 
research.  In  Figure  4.1,  we  highlight  in  gray  the  organizations  we  interviewed  for  this  study, 
which  reside  in  multiple  locations  in  the  broader  Air  Force  structure.  Our  review  of  organiza¬ 
tions  engaged  in  personnel  research  is  not  exhaustive;  rather,  it  was  intended  to  be  illustrative 
of  the  many  organizations  involved.^  The  information  provided  in  this  chapter  is  pulled  from 
our  interviews  with  key  personnel  in  the  various  organizations,  from  information  published  on 


*  Note  that,  shortly  after  this  research  was  completed,  the  Air  Force  began  a  reorganization  process  that  affects  some  of  the 
organizations  discussed  here  (e.g.,  AFPC  and  AFMA).  To  the  extent  that  these  organizational  units  and  subunits  undertake 
the  same  personnel  research-related  activities,  our  findings  still  pertain,  though  organizational  location  and  even  name  may 
change. 

^  We  included  the  most  common  collectors  of  personnel-related  data  that  we  identified.  Other  organizations  may  be 
involved  in  personnel  research,  although  they  are  not  mentioned  here.  However,  our  coverage  provided  us  with  a  broad 
perspective  on  ongoing  endeavors.  Some  of  the  organizations  listed  but  not  interviewed  were  included  for  completeness  as 
examples  but  not  pursued  because  of  our  understanding  that  their  use  of  individual-level  personnel-related  data  was  typi¬ 
cally  not  direct  (i.e.,  they  were  policy  offices). 
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Figure  4.1 

Air  Force  Organizational  Chart  Highlighting  Organizations  Involved  in  Personnel  Research 


NOTE:  FOA  =  field  operating  agency.  AF/A2  =  Intelligence,  Surveillance  and  Reconnaissance.  AF/A3/5  = 
Operations,  Plans  and  Requirements.  AF/A4/7  =  Logistics,  Installations  and  Mission  Support.  AF/A8  =  Strategic 
Plans  and  Programs.  AF/A9  =  Analyses,  Assessments  and  Lessons  Learned.  AF/A10  =  Strategic  Deterrence  and 
Nuclear  Integration  Office. 
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the  Air  Force  portal  or  in  official  Air  Force  publications  (such  as  AFIs),  and  from  institutional 
knowledge  accumulated  by  RAND  employees  over  years  of  working  with  various  Air  Force 
organizations.  Though  the  majority  of  the  information  was  collected  between  2009  and  2010, 
we  did  update  the  information  on  AFRL  by  talking  with  additional  organizational  representa¬ 
tives.  Moreover,  key  personnel  in  the  organizations  we  interviewed  originally  were  given  the 
opportunity  to  review  our  draft  in  2012  in  its  entirety  to  correct  factual  inaccuracies,  update 
information,  and  provide  their  perspective  on  our  conclusions;  however,  some  factual  errors 
may  still  be  present  because  some  organizations  did  not  provide  us  with  a  response. 

We  also  summarize  in  Table  4.1  the  data- collection  methods  of  the  organizations  identi- 
hed,  as  well  as  the  primary  and  secondary  uses  of  personnel  data  of  those  organizations.  Addi¬ 
tional  explanation  of  specific  Table  4.1  entries  can  be  found  in  the  sections  that  follow. 

Air  Education  and  Training  Command 

Headquartered  at  Randolph  Air  Force  Base  (AFB),  Texas,  AETC  is  commanded  by  a  four-star 
general  and  is  one  of  nine  active-duty  MAJCOMs  in  the  Air  Force,  shown  in  Figure  4.1.  AETC 
is  composed  of  the  Air  Eorce  Recruiting  Service  (AERS),  two  numbered  Air  Eorces  (although 


Table  4.1 

Types  of  Personnel  Research  Conducted  by  Air  Force  Organizations 


Organization 

Data- 

Collection 

Method 

JA 

Selection 
or  Person 

Classification  Recruiting 

Training 

Job 

Classification 

Workplace  Feedback, 

Attitudes  and  Evaluation  and 
Perceptions  Promotion 

Manpower 

Competencies  Requirements 

AETC  (MAJCOM) 

OAD  Survey,  focus 

group 


AAD  Archival 


SAS  Archival, 

survey,  test, 
observation 


AFPC  (FOA) 


DSYD  Archival 


DSYA  Archival, 

survey 


SRA  Archival,  C 

survey,  test 


AFMA  (FOA) 


MAS  Archival, 

observation, 
interview, 
focus  group 


Survey 

HO  AF/A1  (Air  Staff) 

AF/A1D  Archival, 

interview 
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Table  4.1 — Continued 


Data- 
Collection 
Organization  Method 


Selection  Workplace  Feedback, 

or  Person  Job  Attitudes  and  Evaluation  and  Manpower 

JA  Ciassification  Recruiting  Training  Classification  Perceptions  Promotion  Competencies  Requirements 


AF/A1P 


AF/A1S 


Archival, 
survey,  test, 
interview, 
focus  group, 
observation 


Survey 


AFMC  (MAJCOM),  AFRL,  711  HPW 


711  HPW/RH  Archival, 

survey,  test, 
focus  group 


USAFSAM  Archival, 

survey,  test, 
interview, 
focus  group 


Other  Various 

Air  Force  methods 

organizations 
(e.g.,  HQ  AF/ 

A9,  AF/A1M) 


NOTE:  Cells  highlighted  in  black  represent  the  organization's  primary  uses  of  personnel  data,  while  cells  highlighted  in  gray  represent  the  organization's  secondary 
uses  of  personnel  data.  "C"  indicates  that  we  know  that  at  least  some  of  the  work  has  been  conducted  by  organizations  external  to  the  Air  Force  (i.e.,  RAND  or  a 
contractor).  Organizations  without  Cs  may  also  rely  on  external  organizations.  "/"  denotes  unknown  information  because  these  organizations  were  not  interviewed. 
Note  that  the  text  includes  information  regarding  some  organizations  we  discovered  at  the  end  of  our  snowball  sampling  period  that  we  did  not  pursue  further.  DSYD 
=  Data  Reports  and  Retrieval  Branch.  DSYA  =  Analysis  Branch.  SRA  =  Strategic  Research  and  Assessment  Branch.  MAS  =  Management  Engineering.  MAPP  =  Performance 
Planning  Branch.  HQ  =  headquarters.  AF/A1D  =  Air  Force  Directorate  of  Force  Development.  AF/A1S  =  Air  Force  Directorate  of  Services.  HPW  =  Human  Performance 
Wing.  711  HPW/RH  =  711  HPW  Human  Effectiveness  Directorate.  USAFSAM  =  U.S.  Air  Force  School  of  Aerospace  Medicine.  AF/A1M  =  Air  Force  Directorate  of  Manpower, 
Organization  and  Resources. 


c/  c/  a  a  a  a  a  a  a 
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19th  Air  Force  is  being  stood  down),  and  Air  University.  AETC  is  in  charge  of  recruiting 
(APRS)  and  providing  the  basic  military  and  technical  training  for  each  career  field  specialty. 
Through  Air  University,  AETC  provides  continuing  professional  and  graduate  education  for 
officers,  enlisted  members,  and  civilians  throughout  their  careers.  AETC  has  an  annual  budget 
of  $8.4  billion  and  an  assigned  workforce  of  nearly  75,000  individuals.  In  fiscal  year  (PY)  2010 
alone,  AETC  was  responsible  for  recruiting  roughly  30,000  individuals  and  providing  basic, 
technical,  and  other  specialized  training  to  roughly  296,000  airmen  (AETC,  2011). 

As  shown  in  Pigure  4.2,  OAD  and  AAD  are  located  under  the  AETC  Headquarters 
Directorate  of  Intelligence,  Operations,  and  Nuclear  Integration,  and  the  SAS  is  located  under 
the  AETC  Headquarters  Directorate  of  Plans,  Programs,  Requirements  and  Assessments 
(AETC/A5/8/9).  OAD  and  AAD  each  report  separately  to  AETC. 

Occupational  Analysis  Division 

This  organization  is  officially  titled  the  OA  flight.^  The  squadron  once  known  as  APOMS 
(described  in  Chapter  Three)  faced  a  significant  downsizing  of  staff  and  a  shift  to  an  all¬ 
civilian  workforce  in  2009.  At  that  time,  its  previous  four  flights  were  consolidated  into  two 

Figure  4.2 

Air  Education  and  Training  Command  Organizational  Chart  Highlighting  Organizations  Involved  in 
Personnel  Research 


RAND  TR1276-4.2 


^  According  to  AETC/Al  subject-matter  experts,  this  organization  is  officially  considered  a  flight  under  an  Air  Force- 
approved  squadron.  It  is  not  officially  considered  a  division.  See  Chapter  Three  for  more  information. 
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new  flights,  one  of  which  is  now  known  as  OAD  (although  OAD  is  the  name  it  now  uses,  it 
is  technically  called  the  OA  flight).  At  the  time  of  our  interview,  OAD  employed  37  civilians. 

OAD’s  primary  focus  is  on  collecting  OA  data  to  inform  the  content  of  enlisted  train¬ 
ing  programs  and  develop  WAPS  test  items.  The  principal  method  of  collecting  OA  data  is 
through  joh  inventories  (JIs),  which  are  administered  to  every  person  in  a  given  enlisted  AFS 
on  a  three-year  cycle.  JIs  for  officer  AFSs  are  conducted  on  request;  although  requests  for  offi¬ 
cer  JIs  are  still  relatively  rare,  they  are  becoming  more  frequent.  From  the  JIs,  OAD  produces 
OA  reports  (OARs),  which  are  automatically  provided  to  CFMs,  SKT  development  teams, 
training  managers,  specialty  training  requirement  teams,  and  U&TWs. 

Besides  describing  job  duties,  OAD  also  administers  separate  surveys  to  a  sample  of  par¬ 
ticipants  to  establish  task  learning  difficulty,  training  emphasis,  and  importance  for  promotion 
tests  (though  this  last  type  of  survey  has  now  been  discontinued  and  the  information  obtained 
via  other  data).  The  task  learning  difficulty  survey  goes  to  a  small,  targeted  sample  of  senior 
noncommissioned  officers  (SNCOs)  (about  100  E-6s  and  E-7s,  depending  on  career  held  size) 
to  determine  how  difficult  it  is  to  train  the  task  effectively;  the  training  emphasis  survey  also 
goes  to  a  similarly  targeted  sample  to  determine  how  important  a  task  is  for  performance  by 
first-term  airmen  (and  hence  should  be  emphasized  in  training);  and  the  testing  importance 
survey  goes  to  SMEs  and  others  in  a  supervisory  capacity  (AFI  36-2623,  2006).  The  survey 
results  are  used  by  CFMs  and  training  managers  at  technical  training  schools  for  selection  of 
tasks  to  be  trained  in  entry-level  training  courses  and  to  assist  SKT  development  teams. 

OARs  are  delivered  to  technical  training  schools  and  briefed  to  users  of  the  data  in  train¬ 
ing,  career  held  management,  and  testing.  JI  data  are  used  to  develop  WAPS  tests.  According 
to  AFI  36-2623  (2006),  OA  information  is  used  for 

•  classifying  AFSs 

•  developing  and  sustaining  Air  Force  training  programs 

•  guiding  utilization  of  personnel 

•  supporting  promotion  tests  for  the  WAPS  and  the  SNCO  Promotion  Program. 

Each  JI,  now  collected  through  an  online  questionnaire,  asks  about  background  infor¬ 
mation  (e.g.,  AFS,  job  satisfaction,  deployments,  reenlistment  intentions,  supervisory  duties; 
additional  demographic  information  is  obtained  from  the  respondent’s  common  access  card 
[CAC])^  and  duty-tasks — a  checklist  of  all  tasks  that  could  be  carried  out  in  performing  the  job 
being  surveyed,  as  well  as  the  amount  of  time  spent  on  the  task. 

The  questionnaire  development  process  takes  a  few  months  for  each  AFS.  It  starts  with 
confirming  and  updating  the  previous  survey  items  and  identifying  new  items  to  add.  This 
process  includes  reviewing  prior  JIs,  job  classification  descriptions,  and  technical  training  stan¬ 
dards;  consulting  with  AFPC,  functional  managers,  the  AETC  pipeline  managers,  and  tech 
training  representatives;  and  conducting  focus  groups  with  SMEs. 

An  OAR  is  prepared  to  describe  the  results  of  all  the  above  analyses,  and  survey  results  are 
presented  to  the  U&TWs  to  provide  insights  into  the  relevance  of  current  training  programs. 

OAD  has  been  asked  to  host  other  surveys  and  will  do  so  if  the  surveys  are  training- 
related,  although  clients  may  have  to  wait  several  months  for  the  survey  to  be  administered. 
External  and  special  studies  conducted  by  OAD  include  gathering  recruiting  and  training 


^  The  JIs  have  been  CAC-enabled  for  about  two  years. 
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input  for  APRS,  enhancing  dental  career  structuring  for  the  Air  Force  Dental  Service,  evalu¬ 
ating  attitudes  toward  mohile  learning  delivery  for  Sheppard  AFB  Mobile  Learning,  and  pro¬ 
viding  training  course  evaluations.  OAD  has  conducted  a  small-husiness  innovative  research 
survey  and  factory  development  surveys  and  would  like  to  develop  surveys  that  capture  joh 
competencies.  OAD  would  like  to  “widen  the  OA  lane”  hut  does  not  intend  to  expand  into 
unrelated  areas. 

OAD’s  role  is  one  of  operational  data  collection  focused  on  supporting  Air  Force  training 
and  testing  programs  hy  administering  and  analyzing  the  JI.  Thus,  although  OA  data  would 
he  useful  in  many  other  areas,  OAD  focuses  only  on  those  areas  outlined  in  its  mission.  Fur¬ 
thermore,  it  has  limited  resources  and  expertise  for  broadening  its  role  and  for  analyzing  its 
own  OA  data. 

OAD’s  role  in  personnel  research  is  narrow  in  focus.  It  supplies  raw  data  from  the  survey 
to  the  standard  OA  data  users,  but  its  staff  does  not  perform  any  sophisticated  statistical  analy¬ 
ses  because  of  limited  resources  and  expertise  to  handle  nonstandard  requests.  For  example, 
a  request  for  a  classified  study  has  been  tabled  for  two  years  because  OAD  is  not  equipped  to 
handle  classified  studies.  As  another  example,  data  reports  include  basic  summary  statistics 
(such  as  sample  sizes  and  average  responses)  for  AFS  subgroups,  but  OAD  does  not  conduct 
tests  of  statistical  significance  to  determine  which  group  differences  are  larger  than  would  be 
expected  by  chance  alone.  Nevertheless,  OAD  staff  does  occasionally  conduct  special  stud¬ 
ies  on  personnel-related  issues;  for  example,  it  is  conducting  research  on  using  the  Emotional 
Quotient  Index  (EQ-i)  as  a  screening  device,  but  such  research  is  not  the  main  focus  of  the 
organization. 

For  this  reason,  in  Table  4.1,  we  show  OAD’s  primary  uses  of  personnel  data  for  JA,  train¬ 
ing  evaluation,  and  job  classification,  with  secondary  uses  including  selection  (as  in  the  case  of 
the  EQ-i)  and  in  recruiting  and  attitudes  (which  are  covered  in  the  additional  assessments  of 
reenlistment  intentions  and  attitudes  also  included  on  the  JI).  Although  OAD’s  data  are  not 
designed  to  be  used  in  other  aspects  of  personnel  research,  they  should  be.  See  Chapter  Two 
for  the  myriad  potential  uses  for  similar  data.  However,  using  its  data  for  some  other  purposes 
would  require  some  additions  to  the  current  JI  content.  (See  the  discussion  on  the  types  of  data 
needed  by  AFMA/MAS  below  for  an  example.) 

Airman  Advancement  Division 

This  is  officially  TXTE.^  The  other  flight  created  after  the  downsizing  of  AFOMS  in  2009 
is  known  as  AAD  (although  AAD  is  the  name  it  now  uses,  it  is  technically  called  the  TXTE 
flight).  AAD’s  mission  is  to  develop  and  distribute  the  promotion  tests  and  study  guides  to 
support  the  WAPS  and  SNCO  Promotion  System,  along  with  the  accompanying  Professional 
Development  Study  Guides.  AAD  develops  three  promotion-testing  tools:  the  SKTs,  the  Pro¬ 
motion  Fitness  Exams  (PFEs),  and  the  U.S.  Air  Force  Supervisory  Exams  (USAFSEs).  Over 
about  18  months,  AAD  develops  275  tests.  Data  from  OAD’s  OARs  are  used  to  inform  the 
content  of  the  test  items. 

AAD’s  staff  is  skilled  at  implementing  the  procedures  established  by  AFHRE  and  its  heirs 
years  ago;  however,  none  have  the  psychometric  expertise  to  continually  adjust  their  techniques 


^  According  to  AETC/Al  subject-matter  experts,  this  organization  is  officially  considered  a  flight  (called  TXTE)  under  an 
Air  Force-approved  squadron.  It  is  not  officially  considered  a  division  (see  Chapter  Three  for  more  information).  Elowever, 
to  distinguish  it  from  the  similarly  named  flight  described  in  the  following  section,  we  refer  to  this  flight  as  AAD. 
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as  professional  practice  changes.  AAD’s  staff  (many  of  whom  have  a  master’s-level  education) 
includes  programmers,  database  managers,  test  psychologists,  and  administrators.  At  the  time 
of  our  interviews,  AAD  had  no  Ph.D.’s  on  staff  and  no  one  with  an  extensive  background  in 
statistics  and  psychometrics. 

Although  AAD’s  staff  uses  personnel  data  heavily,  AAD  does  not  appear  to  be  collect¬ 
ing  any  of  its  own  personnel  data.  It  does  also  receive  statistics  on  the  items  administered  for 
the  WAPS  tests  from  AFPC,  but  again  it  receives  rather  than  produces  the  data.  We  therefore 
marked  AAD’s  primary  use  of  the  personnel  data  under  the  heading  of  evaluations  and  pro¬ 
motions  in  Table  4.1;  however,  AAD  is  best  described  as  a  consumer  rather  than  a  producer  of 
personnel  research. 

Studies  and  Analysis  Squadron 

SAS  was  established  more  than  a  decade  ago  and  serves  as  AETC’s  focal  point  for  quantitative 
analysis,  technology  assessment,  and  operational  test  and  evaluation  (note  that  the  function 
itself  existed  before  this  specific  organization).  SAS’s  mission  is  to  enhance  AETC  recruiting, 
training,  education,  and  decisionmaking  by  conducting  research  on  AETC  training  systems 
and  on  new  ideas  and  technologies.  At  the  time  of  our  interviews,  SAS  was  composed  of  four 
flights:  (1)  the  Test  and  Evaluation  Flight  focuses  on  operational  test  and  evaluation  of  aircrew 
training  systems  and  computer  systems,'^  (2)  the  Technology  Innovation  Flight  assesses  train¬ 
ing  and  education  technology,  (3)  the  Training  Analysis  Flight  assesses  maintenance  training 
requirements  for  new  weapon  systems,  and  (4)  the  Command  Studies  Flight  supports  command 
decisionmaking  through  analysis  and  evaluation  of  AETC  training  systems  and  programs. 

Of  the  four  flights,  the  Command  Studies  flight  is  the  only  one  we  identified  during 
our  interviews  as  regularly  conducting  studies  related  to  personnel  research,  most  of  which  are 
focused  on  training.  However,  the  studies  are  quite  varied,  and  SAS  regularly  employs  a  vari¬ 
ety  of  analytic  approaches  and  data-collection  techniques,  as  needed.  The  Command  Studies 
flight  includes  one  programmer,  three  operations  researchers  with  M.A.’s,  one  civilian  opera¬ 
tions  researcher,  and  a  lieutenant  with  an  operations  research  background.  More  than  half  the 
members  of  the  flight  hold  advanced  academic  degrees.  Staff  members  usually  come  from  the 
Air  force  Institute  of  Technology  (Af  IT)  and  stay  at  SAS  for  three  to  four  years. 

Study  requests  are  typically  directed  to  the  commander,  who  then  distributes  the  work 
based  on  individual  expertise  within  the  flight.  SAS’s  goal  is  to  produce  data  once  or  twice 
and  then,  if  regular  data  collection  is  needed,  pass  responsibility  on  to  the  client  to  continue  to 
implement.  The  following  list  describes  some  of  the  different  projects  SAS  reported  undertak¬ 
ing,  illustrating  the  diversity  of  topics  and  clients: 

•  development,  validation,  and  administration  of  the  Test  of  Basic  Aviation  Skills  (TEAS); 
SAS  provides  the  scores  to  the  pilot  selection  board  (requested  by  AETC  Directorate  of 
Intelligence,  Operations,  and  Nuclear  Integration  [HQAETC/A2/3/10]) 

•  Undergraduate  Pilot  Training  Smooth  flow  Scheduling  system:  SAS  developed  and  tested 
the  scheduling  model  for  air  and  space  basic  course  and  flight  screening  and  is  currently 
transferring  it  to  AfPC  for  ongoing  implementation  (requested  by  HQAETC  A2/3/10) 


^  This  is  the  same  name  given  to  the  flight  described  in  the  previous  section.  We  refer  to  this  flight  as  the  Test  and  Evalu¬ 
ation  Flight. 
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•  analysis  of  new  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  composite  scoring 
method  and  development  of  job  spin  algorithm  (requested  by  AFPC,  AF/Al,  AFRS,  and 
2nd  Air  Force  [2AF]) 

•  Technical  Training  Attrition  Analysis:  regression  analysis  of  data  from  129,000  recruits 
to  study  impact  of  lowering  ASVAB  cutoff  scores  (i.e.,  increasing  the  candidate  pool)  on 
technical  training  attrition  rates.  Study  started  at  AFOMS  but  transferred  to  SAS  because 
AFOMS  did  not  have  operations  research  staff.  SAS  liaised  with  AFRS  and  AFPC  during 
the  study.  (Results  were  presented  to  AFRS  and  2AF.) 

•  Randolph  AFB  automobile  trafhc  control  strategy 

•  Wing  Level  Maintenance  Assessment  Survey:  administered  to  and  analyzed  results  for 
19,000  personnel  (requested  by  AF/A4/7) 

•  balanced  scorecard  study  survey:  SAS  administered  the  survey  designed  to  help  build 
a  collaborative  AETC  team  to  6,000  AETC  personnel  and  analyzed  survey  results 
(requested  by  HQ  AETC) 

•  creation  of  a  tool  to  assign  AESs  to  enlisted  airmen  (requested  by  AETC  Directorate  of 
Plans,  Programs,  Requirements  and  Assessments  [HQ  AETC  A5/8/9]) 

•  analysis  of  the  correlation  between  EQ-i  training  and  success  for  students  entering  sur¬ 
vival,  evasion,  resistance,  and  escape  (SERE)  instructor  training  (requested  by  AEOMS) 

•  evaluation  of  memory  training  program 

•  examination  of  the  adverse  demographic  impact  of  a  new  form  of  the  AEOQT  for  AE PC. 

Although  this  list  shows  several  non-AETC  clients,  most  of  SAS’s  projects  come  through 
AETC.  Some,  such  as  the  training  scheduling  and  EQ-i  studies,  have  even  been  conducted  for, 
or  in  tandem  with,  AEOMS  (presently  OAD).  SAS  personnel  indicated  that  they  also  regularly 
conduct  studies  for  AE RS  and  AEPC.  As  specified  in  its  mission,  SAS’s  primary  use  of  person¬ 
nel  research  is  to  support  training;  however,  because  SAS  assists  with  other  types  of  studies, 
we  also  noted  selection  and  attitudes  as  secondary  uses  of  its  personnel  research  in  Table  4.1. 

The  availability  of  staff  and  resources  is  one  limiting  factor  in  SAS’s  ability  to  conduct 
research.  Eor  example,  HQ  AERS  recently  asked  SAS  whether  it  could  conduct  six  different 
studies,  but  SAS  could  staff  only  one.  When  SAS  is  overwhelmed  with  work,  it  refers  requests 
to  contractors  or  asks  the  client  to  table  the  request  until  a  later  time.  Another  limiting  factor 
is  its  lack  of  expertise  in  certain  aspects  of  personnel  research,  such  as  survey  design.  Never¬ 
theless,  SAS  has  accepted  requests  for  survey  research,  and,  in  those  instances,  it  did  what  it 
could  to  make  the  surveys  successful.  Eor  example,  for  the  Wing  Eevel  Maintenance  Assess¬ 
ment  Survey,  the  client  (AE/A4/7)  composed  the  questions  while  SAS  provided  guidance,  and, 
for  the  Balanced  Scorecard  study,  SAS  administered  the  survey  but  sought  the  assistance  of  a 
professor  at  Air  University  to  develop  the  stratified  random  sample. 

Air  Force  Personnel  Center 

Headquartered  at  Randolph  AEB,  Texas,  AEPC  is  a  EOA  of  HQUSAE,  Deputy  Chief  of  Staff 
for  Manpower  and  Personnel  (AE/Al).  It  is  led  by  a  two-star  general  and  employs  more  than 
2,400  military,  civilian,  and  contract  personnel  (U.S.  Air  force,  2010a).  AfPC’s  main  mission 
is  to  ensure  that  “commanders  around  the  world  have  the  right  number  of  skilled  Air  force 
people  in  the  proper  grades  and  specialties  to  complete  their  missions”  (U.S.  Air  force,  2012). 
In  support  of  this,  AEPC  is  responsible  for  all  personnel  support  and  programs  for  officers  (0-5 
and  below),  enlisted  (E-8  and  below),  and  civilians  (GS-15  and  below).  This  includes  managing 
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personnel  distribution  and  career  progression  and  personnel  programs  focused  on  promotions, 
performance  evaluations,  education,  separations,  and  retirement,  among  others.  AFPC  is  also 
responsible  for  maintaining  all  personnel  data  systems  for  military  and  civilian  personnel  (U.S. 
Air  Force,  2010a). 

AFPC  is  composed  of  seven  directorates: 

1.  Directorate  of  Staff 

2.  Directorate  of  Air  and  Space  Expeditionary  Force  Operations 

3.  Directorate  of  Assignments 

4.  Directorate  of  Civilian  Force  Integration 

5.  Directorate  of  Personnel  Data  Systems 

6.  Directorate  of  Personnel  Services 

7.  Total  Force  Center. 

As  shown  in  Figure  4.3,  AFPC’s  main  personnel  research  activities  are  in  the  Directorate 
of  Staff’s  Research  Analysis  and  Data  Division,  which  is  composed  of  four  branches.  At  the 
time  of  our  interviews,  the  Standards  and  Evaluation  Branch  was  only  in  the  process  of  being 
developed;  however,  the  remaining  three  branches  were  involved  in  personnel  data  collection 
or  analysis  in  some  capacity.  Therefore,  we  describe  those  three  branches  and  their  research 
activities  in  more  detail  below. 

Data  Reports  and  Retrieval  Branch 

DSYD  is  responsible  for  maintaining  more  than  4,500  Air  Force  personnel  data  sets.  These 
include  extracts  from  the  Military  Personnel  Data  System  (MIEPDS),  which  contain  detailed 
past  personnel  records  on  all  Air  Force  active-duty,  reserve,  and  guard  members,  including 
source  of  commission,  date  of  entry,  pay  grade,  career  field,  administrative  actions,  education 
prior  to  joining  the  military,  PME,  bonus  pay  (such  as  the  foreign  language  proficiency  bonus). 

Figure  4.3 

Air  Force  Personnel  Center  Organizational  Chart  Highlighting  Organizations  Involved  in 
Personnel  Research 
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base  location,  race,  gender,  age,  and  other  personal  demographics.  In  addition  to  MILPDS 
extracts,  DSYD  also  maintains  records  from  the  civilian  personnel  database.  All  together,  the 
data  sets  managed  by  this  branch  can  be  pieced  together  to  provide  highly  detailed  longitudi¬ 
nal  information  across  a  wide  range  of  personnel  variables. 

From  these  data  sets,  DSYD  produces  more  than  400  reports  per  year  for  various  Air 
Force  organizations  that  range  from  weekly  or  monthly  data  snapshots  to  quarterly  or  annual 
reports.  Reports  cover  such  topic  areas  as  strength  accounting,  requests  under  the  Freedom  of 
Information  Act  (Pub.  L.  89-554,  1966),  manning  products,  and  retention  statistics.  Recipi¬ 
ent  organizations  may  use  the  reports  to  inform  a  variety  of  personnel-related  topics,  including 
selection,  recruiting,  training,  and  evaluation  and  promotion.  Additionally,  DSYD  conducts 
ad  hoc  data  retrievals  and  reports  when  requested.  For  example,  if  someone  in  the  Air  Staff 
wants  information  on  all  the  pilot-qualified  officers  serving  in  joint  billets  from  the  Air  Force, 
this  branch  would  be  responsible  for  compiling  the  data.  Other  ad  hoc  report  topics  include 
requests  for  demographics,  assignments,  and  PME  profiles.  Because  DSYD’s  personnel  data 
are  used  to  inform  any  number  of  research  areas,  we  noted  those  areas  as  primary  uses  in 
Table  4.1.  However,  DSYD  does  not  do  any  in-depth  analysis  and  interpretation  of  the  topic 
areas  in  Table  4.1;  it  only  provides  data  and  summary  reports  to  be  used  by  other  organizations. 

Data  requests  commonly  come  from  senior  leaders  in  the  U.S.  Department  of  Defense 
(DoD),  the  offices  of  the  Secretary  of  the  Air  Force,  Air  Staff,  and  AFPC.  DSYD  also  provides 
data  to  other  DoD-wide  agencies,  such  as  the  Defense  Manpower  Data  Center  (DMDC), 
which  maintains  personnel  data  records  on  all  military  services,  and  the  Defense  Logistics 
Agency  (DLA),  which  provides  logistics  support  for  all  of  DoD,  the  U.S.  Department  of  Trans¬ 
portation  (DOT),  and  the  Defense  Finance  and  Accounting  Service  (DFAS). 

Analysis  Branch 

DSYA  is  responsible  for  producing  any  personnel  reports  that  require  more-complicated  calcu¬ 
lations  beyond  the  simple  requests  for  assembling  of  data  sets  normally  handled  by  DSYD.  As 
shown  in  Table  4.1,  it  conducts  studies  and  policy  reviews  on  such  areas  as  selection,  recruiting, 
training,  and  promotion  (including  providing  support  for  the  WAPS).  Overall,  these  analyses 
focus  on  providing  information  about  the  current  or  historical  state  of  personnel  in  the  Air 
Force.  For  example,  DSYA  reported  that  recent  projects  have  included  examining  the  impact 
of  recruiting  waivers,  examining  the  effect  of  PME  credits  on  promotion  and  school  selection 
rates,  examining  differences  in  retention  rates  by  demographics,  and  creating  a  Reserve  Officer 
Training  Corps  (ROTC)  AES  classification  model.  It  also  described  some  minor  survey  devel¬ 
opment  work  to  measure  raters’  perceptions  of  the  past  performance  of  individuals  recalled 
from  the  guard  and  reserves.  Therefore,  we  have  also  highlighted  workplace  attitudes  and  per¬ 
ceptions  as  a  secondary  use  of  the  data  DSYA  collects.  Overall,  the  reports  produced  address 
requests  from  within  AEPC,  as  well  as  from  other  organizations  in  the  Air  force  (e.g.,  AETC 
or  Headquarters  U.S.  Air  force  [HAL])  or  DoD. 

Strategic  Research  and  Assessment  Branch 

Housed  within  AEPC,  SRA  acts  as  a  liaison  with  HAP  force  Management  Policy  Division 
(AE/AIPE)  to  identify  and  oversee  personnel-related  research  efforts  in  the  Air  force. ^  SRAs 


^  At  the  time  of  the  interviews  with  SRA,  SRA  liaised  with  AF/AIPF,  the  Force  Management  Policy  Division.  This  was 
later  renamed  AF/AIPT. 
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mission  was  created  as  part  of  a  restructuring  of  AFPC  in  2007,  with  the  main  goal  of  fill¬ 
ing  in  key  research  gaps  remaining  after  the  dissolution  of  AFHRL  and  its  successors,  such 
as  Armstrong  Laboratory,  Manpower  and  Personnel  Research  Division  (AL/HRM),  although 
individuals  within  the  unit  were  performing  similar  functions  within  AFPC  previously.  How¬ 
ever,  because  SRA  has  a  limited  number  of  staff,  these  studies  are  generally  conducted  by  con¬ 
tractors  or,  in  some  cases,  other  Air  Force  organizations  (e.g.,  SRA  supervised  the  validation 
of  the  TEAS  by  SAS).  It  described  several  past  and  current  personnel  research  efforts  (some 
conducted  in-house  and  others  contracted  to  outside  agencies).  Examples  of  the  types  of  topics 
are  shown  here;  given  these  examples,  we  note  several  corresponding  primary  uses  of  personnel 
data  in  Table  4.1: 

•  AFS  restructuring 

•  the  officer  accession  system 

•  the  development  of  job  performance  measures 

•  test  development  and  validation  (e.g.,  AFOQT) 

•  identification  and  validation  of  screeners  to  improve  person-job  matching  in  training 

•  development  of  a  selection  battery  for  operators  of  remotely  piloted  aircraft  (RPAs) 

•  development  of  realistic  job  previews 

•  development  of  new  ASVAB  composite  scores  for  use  in  classification 

•  development  and  maintenance  of  the  Human  Resources  Research  Databank  (HRRD), 
which  serves  as  a  historical  archive  of  more  than  50  years  of  personnel-related  research 
data. 

SRA  is  also  responsible  for  fielding  issues  with,  or  proposed  improvements  to,  the  person¬ 
nel  testing  system  (AFI  36-2605,  2008)  for  the  Directorate  of  Force  Management  Policy,  Force 
Management  Division  (AF/AIPF).  This  includes  reviewing  proposed  research  related  to  per¬ 
sonnel  testing  changes,  providing  advice  on  the  research  plan  and  Institutional  Review  Board 
(IRB)  procedures,  and  reviewing  final  validation  efforts  and  results  to  ensure  they  are  consis¬ 
tent  with  well-established  professional  guidelines  for  testing  and  assessment  (e.g.,  American 
Educational  Research  Association,  American  Psychological  Association,  and  National  Council 
on  Measurement  in  Education,  1999).  After  review,  a  package  of  proposed  changes  is  then  for¬ 
warded  to  AE/AIPE  for  approval.  SRA  also  oversees  testing  operations  for  the  AE OQT  (used 
in  officer,  pilot,  and  combat  system  operator  selection)  and  the  TEAS  (used  in  pilot  selection). 
SRA  also  described  collecting  JA-type  data  through  contractors  as  part  of  its  research  efforts; 
however,  as  shown  in  Table  4.1,  this  is  not  a  primary  data-collection  activity  for  this  branch. 

Many  of  the  other  organizations  we  interviewed  were  not  aware  that  this  branch  exists  or 
that  it  is  available  for  consultation.  Instead,  most  of  SRAs  clients  are  identified  when  the  branch 
hears  of  Air  Eorce  testing  efforts,  which  require  SRAs  approval  (as  specified  in  AEI  36-2605, 
2008). 

Air  Force  Manpower  Agency 

Eike  AE  PC,  AEMA  is  a  EOA  headquartered  at  Randolph  APB,  Texas.  APMA  reports  to  AP/ 
AIM.  Eed  by  a  colonel  (0-6),  APMA  employs  a  total  of  425  active-duty  and  civilian  person¬ 
nel  (“2009  USAP  Almanac,”  2009)  to  accomplish  its  primary  mission:  providing  “the  tools  to 
identify  essential  manpower”  needs  (U.S.  Air  force,  2010b,  5  2).  Among  its  main  responsibili¬ 
ties  are 
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•  determining  manpower  requirements,  standards,  and  programming  and  resourcing  fac¬ 
tors  (i.e.,  how  long  it  takes  for  an  average  worker  to  do  a  task  and  how  much  time  a  person 
is  available  to  do  it) 

•  creating  and  maintaining  standard  position  descriptions 

•  conducting  and  overseeing  Air  Force  attitude  and  opinion  surveys 

•  providing  civilian  classification  oversight  and  centralized  operational  classification. 

As  shown  in  Figure  4.4,  AFMA  is  made  up  of  four  divisions — Commercial  Services  Man¬ 
agement  (MAC),  Central  Civilian  Classification  (MAH),  Management  Engineering  (MAS), 
and  Performance  Management  (MAP) — and  five  regionally  based  Manpower  Requirements 
Squadrons  (MRSs),  whose  primary  responsibility  is  to  quantify  manpower  requirements.  MAS 
and  MAP  are  the  two  divisions  we  identified  as  regularly  conducting  personnel-related  research 
and  data- collection  activities. 

Management  Engineering  Division 

MAS  is  responsible  for  establishing  the  manpower  standards — or  estimates  for  the  number  of 
man-hours  necessary  to  complete  a  given  task — for  all  Air  Force  career  fields,  which  are  used 
to  develop  estimates  of  the  amount  of  manpower  necessary  for  a  given  career  field  (i.e.,  man¬ 
power  requirements). 

The  MRSs  conduct  the  manpower  studies,  with  oversight  by  MAS  senior  staff.  Each 
MRS  has  about  50  staff,  half  military  and  half  civilian,  including  management  analysts,  indus¬ 
trial  engineers,  and  computer  scientists.  Interestingly,  none  have  backgrounds  in  psychology 
or  operations  research,  two  fields  that  would  also  seem  relevant  to  the  issue  of  determining 
manpower  requirements. 

Figure  4.4 

Air  Force  Manpower  Agency  Organizational  Chart  Highlighting  Organizations  Involved  in  Personnel 
Research 
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Using  available  data,  base-visit  observations,  and  SME  judgment,  the  MRSs  work  to 
identify  the  range  of  activities  in  a  given  career  field  and  the  typical  time  it  takes  to  execute 
each  activity.  Although  existing  data  sources  are  considered  in  the  process  of  establishing  man¬ 
power  standards,  such  data  are  often  not  available  to  AFMA  (e.g.,  OA  data  are  not  regularly 
provided  to  MAS)  or  do  not  directly  address  their  needs  (e.g.,  MILPDS  data  do  not  contain 
elements  useful  for  developing  manpower  standards,  and  OA  data  do  not  include  estimates  for 
how  long  it  takes  the  average  worker  to  complete  a  task);  hence,  SME  judgment  and  observa¬ 
tion  usually  have  the  greatest  impact  on  the  final  standards.  According  to  the  MAS  representa¬ 
tives  we  interviewed,  collecting  the  necessary  data  is  not  hard;  finding  the  data  and  knowing 
someone  who  has  them  is  much  harder.  When  a  manpower  study  is  complete,  a  summary 
report  of  about  ten  to  12  pages  is  posted  on  the  AFMA  website. 

MAS’s  (and  the  subordinate  MRSs’)  role  in  collecting  and  analyzing  job  requirements 
puts  its  work  squarely  in  the  realm  of  personnel  research.  Table  4.1  shows  the  primary  purpose 
of  its  research  is  describing  the  job  task  duration  requirements  (e.g.,  a  form  of  JA)  and  man¬ 
power  analyses. 

Performance  Management  Division 

The  Performance  Planning  Branch  (MAPP)  within  MAP  includes  the  Air  Force  Survey  office. 
MAPP’s  primary  responsibility  in  this  regard  is  overseeing  all  Air  Force— wide  assessments  of 
attitudes  or  opinions  (i.e.,  including  any  polls,  surveys,  questionnaires,  interviews,  or  focus 
groups).  AFMA  assigns  official  survey  control  numbers  (SCNs),  which  designate  that  an  assess¬ 
ment  has  been  officially  approved  by  AFMA.®  Air  Force  personnel  are  asked  to  ignore  any  Air 
Force-wide  studies  of  opinions  or  attitudes  not  approved  by  AFMA. 

As  part  of  the  approval  process,  MAPP  ensures  that  each  survey 

•  uses  sound  measurement  practices  (i.e.,  will  yield  reliable  and  valid  results) 

•  meets  operations  security  requirements 

•  satisfies  the  Privacy  Act  (Privacy  Act  of  1974,  Pub.  F.  93-579)  where  applicable 

•  does  not  create  an  undue  survey  burden  on  personnel 

•  is  endorsed  by  Air  Force  leadership  to  which  the  survey  findings  apply  (to  reduce  duplica¬ 
tion  of  effort). 

MAPP  estimates  that  it  reviews  about  80  percent  of  all  surveys  of  Air  Force  personnel.^ 
For  the  three  months  from  January  to  March  2010,  it  reported  reviewing  46  survey  requests, 
of  which  28  were  approved  and  18  were  rejected.  Approved  surveys  included  such  topics  as  per¬ 
sonal  safety;  the  preparations  for,  exposures  to,  and  impacts  of  Air  Force  deployments;  mental 
health;  and  several  personnel  system  surveys.  Examples  of  the  types  of  studies  approved  by 
MAPP  are 

•  tobacco  use  survey 

•  tobacco  use,  postban  survey 


®  Other  organizations,  including  the  Inspector  General,  Air  University,  and  AFIT,  can  issue  SCNs  or  provide  other  forms 
of  approval  for  assessments  of  attitudes  or  opinions.  Air  University  and  AFMA  coordinate  survey  authorizations,  while 
AFIT,  which  issues  SCNs  for  student  projects,  operates  more  independently. 

^  Among  the  remaining  20  percent  are  surveys  approved  by  other  organizations,  such  as  AFIT  and  AF/AIPF,  and  surveys 
that  are  not  Air  Force-wide,  such  as  those  administered  to  a  single  base  and  that,  thus,  do  not  require  AFMA  approval. 
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•  Air  Force  personal  safety  survey 

•  preparations,  exposures,  and  impacts  of  Air  Force  deployments:  a  focus  on  mental  health 

•  Air  Force  center  for  engineering  and  the  environment  stakeholder  assessment 

•  total  force  integration  personnel  accounting  symbol  (PAS)  code  deployment  process  task¬ 
ing 

•  advanced  maintenance  and  munitions  officer  school  (ammos)  student  research 

•  balanced  scorecard  survey 

•  employee  exit  interview 

•  resilient  military  couples 

•  workforce  development  and  gap  analysis 

•  performance-based  award  survey 

•  department  of  engineering  mechanics  graduate  and  supervisor  surveys 

•  line  of  duty  and  Medical  Evaluation  Board  survey 

•  mentorship  program;  mentee  preprogram,  mentee  postprogram,  and  mentor  postpro¬ 
gram  evaluations 

•  Air  Force  climate  survey  pilot  test 

•  nonstandard  aviation,  aircrew  survey 

•  AFIT  Squadron  Officer  School 

•  MQ-9  simulator  survey 

•  follow-on  review:  issues  from  Fort  Hood  survey 

•  effects  of  downsizing 

•  Security  Forces:  study  on  deployment 

•  intelligence,  surveillance,  and  reconnaissance  (ISR)  deployment  survey. 

MAPP  also  helps  develop,  administer,  and  analyze  some  surveys  on  a  case-by-case  basis 
as  time,  resources,  and  workload  priorities  permit.  MAPP  has  a  small  number  of  staff,  which 
limits  how  much  assistance  it  can  provide,  although  it  does  have  considerable  analytic  capacity 
well-suited  to  this  purpose,  with  several  behavioral  scientists,  including  some  research  psychol¬ 
ogists,  several  operations  research  analysts,  and  some  information  technology  (IT)  staff.  In  the 
three  months  from  January  to  March  2010,  MAPP  designed  and  implemented  eight  large  and 
11  medium-sized  surveys.  Given  that  MAPP’s  conduct  of  studies  on  attitudes  and  opinions  is  a 
central  part  of  its  mission,  we  marked  its  primary  use  of  personnel-related  data  as  falling  under 
“workplace  attitudes  and  perceptions”  in  Table  4.1. 

Although  MAPP  regularly  collects  or  approves  the  collection  of  very  rich  personnel  data 
sources,  its  data  sharing  and  linking  capability  is  limited,  to  protect  the  confidentiality  of 
the  respondents.  For  example,  one  MAPP-developed  survey,  the  Air  Force  climate  survey,  is 
administered  every  two  or  three  years  and  covers  a  wide  range  of  attitudes,  including  over¬ 
all  job  satisfaction,  perceived  stress  levels,  workloads,  work  hours,  reenlistment  intentions, 
views  about  leadership,  and  views  on  other  pressing  issues.  Although  the  2010  climate  survey 
included  many  of  the  same  topics  as  the  2008  version  (which  included  nearly  250,000  respon¬ 
dents),  identifiers  are  not  retained  after  each  survey  is  completed;  therefore,  only  aggregate 
estimates  by  groups  can  be  compared  longitudinally. 

MAPP’s  role  in  conducting  attitudinal  and  opinion  studies  is  also  limited  because  it  does 
not  have  access  to  data  from  the  many  externally  built  surveys  (or  data  collected  through  other 
means),  even  if  such  surveys  are  MAPP-approved.  Studies  designed  and  analyzed  by  outside 
contractors  or  even  other  organizations  in  the  Air  Force  are  not  required  to  deliver  the  data 
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to  MAPP,  and  MAPP  typically  does  not  have  reason  to  request  them.  MAPP  does,  however, 
direct  inquiries  to  the  owners  of  externally  collected  data  in  the  hopes  of  facilitating  data  shar¬ 
ing,  and  it  does  retain  deidentified  data  from  its  own  studies  and  can  reanalyze  those  data  as 
needed. 

Deputy  Chief  of  Staff,  Manpower,  Personnel  and  Services 

At  HAF,  five  deputy  chiefs  of  staff  (DCSs)  are  responsible  for  plans  and  policies  for  the  major 
operational  activities  of  the  Air  Force:  AF/Al,  Air  Force  Intelligence,  Surveillance  and  Recon¬ 
naissance  (AF/A2),  Air  Force  Operations,  Plans  and  Requirements  (AF/A3/5),  AF/A4/7,  and 
Air  Force  Strategic  Plans  and  Programs  (AF/A8).  They  report  directly  to  the  highest-ranking 
military  officer  in  the  Air  Force,  the  Chief  of  Staff,  and  preside  over  the  HAF  Air  Staff. 

The  DCS  for  AF/Al,  which  is  responsible  for  plans  and  policies  for  managing  military 
and  civilian  personnel  life  cycles  (DoD,  2006),  has  four  directorates  (shown  in  Figure  4.5) 
with  suborganizations  that  were  identified  in  our  interviews  as  frequently  engaged  in  personnel 
research.  Those  suborganizations  are  described  below. 

Air  Force  Directorate  of  Force  Deveiopment 

AF/AID  has  six  suborganizations,  as  shown  in  Figure  4.6.  We  identified  two  suborganiza¬ 
tions  in  AF/AID  as  organizations  that  have  occasions  to  conduct  personnel  research:  Air  Force 
Airman  Development  Division  (AF/Al  DD)  and  Air  Force  Force  Development  Integration 
(AF/AIDI). 

Figure  4.5 

Deputy  Chief  of  Staff  for  Manpower  and  Personnel  Organizational  Chart  Highlighting 
Organizations  Involved  in  Personnel  Research 


NOTE:  DPG  =  Air  Force  General  Officer  Management.  DPO  =  Air  Force  Colonels  Management.  DPE  = 

Chief  Master  Sergeant  Management.  DPS  =  Air  Force  Senior  Executive  Management.  AF/Al  E  =  Executive 
Services.  AF/Al  G  =  Air  Force  Guard  Advisor.  AF/Al  H  =  Air  Force  Reserve  Advisor.  AF/Al  I  =  Air  Force  Action 
Group.  AF/Al 0  =  Air  Force  Equal  Opportunity. 
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Figure  4.6 

Air  Force  Directorate  of  Force  Development  Organizational  Chart 
Highlighting  Organizations  Involved  in  Personnel  Research 


NOTE:  AF/A1 DG  =  Air  Force  Culture,  Region,  and  Language.  AF/A1 DL  =  Air  Force 
Learning.  AF/A1  DO  =  Air  Force  Commissioning  Programs.  AF/A1 DV  =  Air  Force 
Diversity  Operations. 
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Air  Force  Airman  Development  Division 

The  first  organization,  AF/AIDD,  focuses  on  force  development  policies,  including  developing 
the  force-wide  institutional  competencies  list  (AFI  36-2013,  2008).  At  the  time  of  our  inter¬ 
view,  AF/AlDD  described  its  most  recent  personnel  research  effort  as  civilian  talent  manage¬ 
ment.  The  study,  carried  out  by  AF/AIDD  personnel,  focused  on  identifying  the  most-critical 
institutional  competencies  and  occupational  requirements  for  each  senior  executive  service 
(SES)  position  (n  =  165),  and  the  study  findings  were  used  to  adjust  various  aspects  of  SES 
development.  Training  and  competencies  are,  therefore,  noted  as  primary  uses  in  Table  4.1. 

Air  Force  Force  Development  Integration 

The  second  suborganization  involved  in  personnel  research  in  AE/AID,  AE/AIDI,  is  tasked 
with  ensuring  that  candidates  are  qualified  for  senior-officer,  enlisted,  and  civilian  positions. 
One  major  component  of  AE/AlDI’s  work  is  identifying  the  experiences,  skills,  and  competen¬ 
cies  needed  to  be  considered  qualified  for  those  senior  positions,  something  it  does  using  such 
records  as  duty  history,  educational  background,  and  PME.  At  the  time  of  our  interviews, 
AE/AIDI  was  developing  tools  to  help  OEMs  identify  critical  competencies  and  conduct  gap 
analyses  comparing  current  personnel  with  the  personnel  they  need  in  terms  of  experience, 
education,  and  training.  These  activities  are  noted  in  Table  4.1  as  secondarily  involving  JA 
data,  with  the  primary  uses  being  training  and  development  of  competencies. 

AE/AIDI  also  provides  tools  for  career  development.  Eor  example,  one  tool,  developed 
by  an  external  contractor,  allows  an  employee  to  pick  three  peers,  three  subordinates,  and  one 
supervisor  to  provide  feedback  about  his  or  her  strengths  and  development  needs  on  27  insti¬ 
tutional  competencies.  (This  is  noted  as  feedback  in  Table  4.1.)  Some  SNCO  PME  classes  (at 
the  discretion  of  the  instructor)  collect  this  feedback  before  students  go  to  the  schoolhouse,  and 
then  the  results  are  used  during  the  course,  with  instructor  guidance.  In  past  years,  AE/AIDI 
has  also  conducted  other  personnel-related  surveys  (e.g.,  on  institutional  competencies  for  chief 
master  sergeants),  some  of  which  has  been  with  the  assistance  of  RAND. 
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At  the  time  of  our  interviews,  AF/AIDI  was  not  collecting  any  new  data  because  of 
budget  constraints. 

Air  Force  Directorate  of  Force  Management  Policy 

AF/AIP,  which  is  “responsible  for  developing  objectives,  performing  analysis,  and  developing 
force  management  policies  to  effectively  shape  and  balance  the  AF’s  Total  Force”  (AFI  36-2013, 
2013),  has  several  divisions  illustrated  in  Figure  4.7.  At  least  one  of  these  divisions — AF/ 
AlPF — is  heavily  involved  in  personnel  research. 

Air  Force  Force  Management  Division 

AF/AIPF  has  requested  RAND’s  help  in  multiple  personnel  research  efforts,  including  exam¬ 
ining  strength  requirements  in  enlisted  jobs,  improving  the  person-job  matching  system,  eval¬ 
uating  the  AFOQT,  and  improving  officer  accession  selection  processes.  In  addition  to  its 
collaboration  with  RAND,  AF/AIPF  plans  to  administer  a  survey  to  validate  a  set  of  occupa¬ 
tional  competencies  recently  produced  by  two  other  contractors;  ultimately,  it  aims  to  develop 
occupational  competencies  for  all  AFSs.  It  also  described  plans  to  develop  a  web-based  tool  so 
staff  can  provide  self-assessments  of  their  skills  in  the  identihed  competencies  and  supervisors 
can  confirm  the  accuracy  of  the  self-assessments.  In  addition,  in  2009,  AF/AIPF  developed 
the  items,  sample,  and  analysis  plans  for  a  retention  survey,  which  was  then  fielded  and  imple¬ 
mented  by  AFMA.  Finally,  AF/AIPF  regularly  conducts  in-house  surveys  and  workshops  (typ¬ 
ically  focused  on  career  held  retention  or  sustainment)  and  analyzes  their  results.  These  various 
activities  are  noted  in  Table  4.1  as  primary  uses  of  data,  with  a  secondary  use  of  JA  data. 

AF/AIPF  has  a  staff  of  about  a  dozen  analysts,  mostly  officers  and  some  civilians,  but 
it  also  relies  on  the  experience  of  AFPC’s  SRA,  which  works  closely  with  AF/AIPF  to  award 

Figure  4.7 

Air  Force  Directorate  of  Force  Management  Policy  Organizational  Chart 
Highlighting  Organizations  Involved  in  Personnel  Research 


NOTE:  AF/A1  PA  =  Air  Force  Compensation  and  Travel  Division.  AF/A1  PC  = 

Air  Force  Civilian  Policy  Division.  AF/A1  PI  =  Air  Force  Integration  Section  Division. 
AF/A1 PP  =  Air  Force  Military  Force  Policy  Division.  AF/A1  PR  =  Air  Force  Personnel 
Readiness  Division.  AF/A1PS  =  Air  Force  Continuum  of  Service  Division.  AF/A1PT  = 
Air  Force  Training  and  Educational  Requirements  Division. 
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contracts  for  studies  addressing  various  personnel  research  issues.  SRA  and  AF/AIPF  together 
determine  research  agendas,  write  statements  of  work,  monitor  contracts,  brief  results,  and 
make  recommendations.  AF/AIPF  hired  three  new  industrial  and  organizational  psychologists 
in  2011  (physically  located  with  SRA)  in  the  hopes  of  creating  a  technically  trained  in-house 
personnel  research  team  to  reduce  its  reliance  on  contractors. 

Air  Force  Directorate  of  Services 

AF/AIS  is  responsible  for  Air  Force-wide  and  base-level  service  programs,  including  lodg¬ 
ing,  food  service,  htness,  child  development  programs,  golf,  social  clubs,  libraries.  Air  Force 
uniforms,  sexual  assault  prevention,  airman  and  family  readiness,  mortuary  affairs,  recreation 
activities,  and  entertainment  (AFI  36-2013,  2009a).  As  shown  in  Figure  4.8,  AF/AIS  has  nine 
separate  divisions  or  agencies. 

Air  Force  Airman  and  Family  Services  Division 

Although  many  of  these  agencies  may  conduct  surveys  to  support  their  missions,  we  learned 
from  our  interviews  that  AF/AISA  has  conducted  at  least  three  major  personnel  research 
surveys — the  Community  Assessment  Survey  (CA),  the  Quality  of  Life  Survey  (QOL),  and  a 
Spouse  Survey — all  of  which  are  designed  to  assess  various  attitudes  and  perceptions  (as  noted 
in  Table  4.1). 

The  CA,  produced  in  conjunction  with  the  Community  Action  Information  Board 
(CAIB),  has  evaluated  topics  of  interest  to  each  Air  Force  community  (i.e.,  active  duty,  reserve, 
guard,  spouses,  and  civilians)  every  two  to  three  years,  since  1993.  Although  AFMA  devel- 

Figure  4.8 

Air  Force  Directorate  of  Services  Organizational  Chart  Highlighting 
Organizations  Involved  in  Personnel  Research 


NOTE:  AF/AISA  =  Air  Force  Airman  and  Family  Services  Division.  AF/A1SFR  = 

Air  Force  Sexual  Assault  Prevention  Response.  AF/A1SO  =  Air  Force  Combat 
Operations  Division.  AF/A1SS  =  Air  Force  Airman  Resiliency  Program  Division. 
AF/A1SR  =  Air  Force  Requirements  Division.  AF/A1ST  =  Air  Force  Armed  Forces 
Entertainment  Division.  AF/A1SZ  =  Air  Force  Warrior  and  Survivor  Care.  AFMAO  = 
Air  Force  Mortuary  Affairs  Operation.  AFSVA  =  Air  Force  Services  Agency. 
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oped,  fielded,  and  analyzed  the  CA  in  the  past,  in  2010,  AFMA  was  unahle  to  assist;  hence,  an 
outside  contractor  was  hired  to  take  on  that  role. 

The  QOL  survey,  which  began  in  2008  and  was  administered  again  in  2010,  is  used  to 
drive  Air  Force-wide  changes  to  base  services.  Designed  by  an  external  consulting  firm,  it 
evaluates  how  various  base-level  programs  (e.g.,  lifestyle,  relationships,  units,  housing,  librar¬ 
ies,  child  and  youth  programs,  fitness  programs)  affect  QOL,  people’s  satisfaction  with  the 
programs,  and  the  programs’  effects  on  retention  and  readiness.  In  2008,  50,000  active-duty 
airmen  were  invited  to  take  the  survey,  and  about  14,000  (28  percent)  responded.  A  larger 
sample,  including  family,  civilians,  guard,  and  reserve,  was  planned  for  the  2010  administra¬ 
tion  to  allow  reporting  of  results  by  multiple  subgroups  (e.g.,  by  AFS,  base,  and  MAJCOM). 

The  Spouse  Survey,  administered  in  2009,  was  a  one-time  event  to  supplement  informa¬ 
tion  from  an  earlier  survey  on  Air  Force  culture  and  included  questions  on  finances,  children, 
and  satisfaction  with  Air  Force  leadership.  No  future  spouse  surveys  are  planned  because  of 
funding  constraints. 

AF/AlSA’s  personnel  research  results  have  been  briefed  to  a  variety  of  Air  Force  leaders, 
including  the  Chief  of  Staff,  AF/AIS,  mission  support  group  training  commanders,  and  the 
functional  community.  When  reports  are  published,  they  are  accessible  to  base-level  com¬ 
mands,  MAJCOMs,  and  mission  support  group  commanders.  Although  AF/AISA  owns  the 
survey  databases,  in  some  cases,  the  data  reside  with  a  contractor  (e.g.,  a  contractor  retains  the 
CA  data). 

Air  Force  Directorate  of  Manpower,  Organization  and  Resources 

AF/AIM  was  among  the  organizations  identified  in  the  AF/Al  Air  Staff  community  that  may 
be  involved  in  personnel  research.  Although  we  did  interview  representatives  from  AFMA  (the 
FOA  described  previously  and  illustrated  in  Figure  4.4),  we  did  not  interview  representatives 
from  the  other  AF/AIM  Air  Staff  suborganizations  mentioned  in  Figure  4.4.  This,  therefore,  is 
an  example  of  an  organization  we  identified  through  our  snowball  sampling  technique  but  did 
not  contact  for  interviews.  Nevertheless,  we  note  it  here  as  having  been  identified  as  potentially 
involved  in  personnel  research,  with  the  primary  area  of  application  of  the  data  in  the  area  of 
manpower  requirements. 

Deputy  Chief  of  Staff,  Studies  and  Analyses,  Assessments  and  Lessons  Learned 

Another  of  the  five  HAF  DCSs  who  report  directly  to  the  Chief  of  Staff — AF/A9 — is  also 
likely  involved  in  personnel  research.  AF/A9’s  vision  (as  described  in  various  A9  briefings)  is 
to  ensure  gold-standard  quality  studies,  analyses,  assessments,  and  lessons-learned  processes; 
to  illuminate  emerging  DoD  issues;  and  to  fireproof  key  Air  Force  leadership  decisions.  It  is 
composed  of  the  following  five  directorates: 

•  Air  Force  Analyses  and  Assessments  (AF/A9A) 

•  Air  Force  Force  Structure  Analyses  (AF/A9F) 

•  Air  Force  Analyses  Foundations  and  Integration  (AF/A9I) 

•  Air  Force  Lessons  Learned  (AF/A9L) 


Our  list  of  organizations  is  not  intended  to  be  comprehensive;  nevertheless,  because  this  particular  organization  may 
regularly  conduct  personnel  research,  we  felt  that  its  absence  here  would  be  conspicuous.  Therefore,  we  opted  to  include  it 
but  provide  only  cursory  details. 
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•  Air  Force  Resource  Analyses  (AF/A9R). 

Although  we  did  not  interview  any  representatives  from  AF/A9,  as  we  did  with  AF/AIM, 
its  involvement  in  personnel  research  was  mentioned  during  our  snowball  technique,  and  per¬ 
sonnel  research  seems  applicable  to  the  areas  dehned  by  its  various  directorates.  For  this  reason, 
we  have  included  a  brief  mention  of  it  here;  because  we  did  not  interview  AF/A9  representa¬ 
tives,  we  are  not  providing  specihc  details  on  current  or  past  personnel  research  activities, 
though  our  understanding  indicates  that  AF/A9’s  primary  use  of  personnel  data  is  for  deter¬ 
mining  manpower  requirements.'' 

Air  Force  Materiel  Command's  Air  Force  Research  Laboratory 

As  shown  in  Figure  4.9,  AFRL  is  a  suborganization  of  one  of  the  Air  Force’s  nine  active-duty 
MAJCOMs,  AFMC.  It  is  located  at  Wright-Patterson  AFB.  AFRL’s  mission  is  “leading  the 
discovery,  development  and  integration  of  affordable  warfighting  technologies  for  America’s 
aerospace  forces”  (U.S.  Air  Force,  2009,  5  1).  AFRL  employs  approximately  1,400  military 
and  5,400  civilian  personnel  throughout  the  Office  of  Scientific  Research  and  nine  technology 
directorates  and  is  responsible  for  the  Air  Force’s  $2  billion  science  and  technology  budget. 

Figure  4.9 

Air  Force  Research  Laboratory  Organizational  Chart  Highlighting 

Organizations  Involved  in  Personnel  Research 


NOTE:  AFOSR  =  Air  Force  Office  of  Scientific  Research.  71 1  HPW/HP  =  71 1  HPW  Human 
Performance  Integration  Directorate. 
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"  This  organization  represents  one  of  the  later  discoveries  of  our  snowball  sampling  technique;  as  a  result,  we  did  not 
include  it  in  our  interviews.  As  a  reminder,  our  list  of  organizations  is  not  intended  to  be  comprehensive;  nevertheless, 
because  this  particular  organization  could  have  a  need  to  conduct  personnel  research  on  a  regular  basis,  we  felt  that,  if  it 
were  omitted  from  this  chapter,  its  absence  would  be  conspicuous.  Therefore,  we  opted  to  include  it  but  provide  only  cursory 
details. 


For  more  information  on  AFRL,  see  U.S.  Air  Force  (2009). 
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When  we  first  started  our  interviews,  we  met  with  a  representative  from  the  711  HPW 
(one  of  the  nine  technology  directorates  in  AFRL  shown  in  Figure  4.9),  who  described  the 
organization  hut  stated  that  the  711th  was  not  engaged  in  any  personnel-related  data  collection 
or  research  activities.  Since  then,  however,  we  have  learned  that  the  711th  is  trying  to  reengage 
in  manpower  and  personnel  science  and  technology  with  a  focus  on  such  topics  as  RPA,  cyher 
personnel,  and  hattlefield  airmen,  and  we  have  spoken  with  some  additional  personnel.  This 
work  is  being  conducted  in  the  Human  Effectiveness  Directorate  and  in  USAFSAM,  two  of 
the  three  organizations  within  the  711  HPW  umbrella.  The  overall  711  HPW  mission  is  “to 
advance  human  performance  in  air,  space,  and  cyberspace  through  research,  education,  and 
consultation”  (AFRL,  201  Id,  p.  1). 

AFRL’s  work  in  the  training  domain  is  an  ongoing  effort  dating  to  prior  to  AFHRL’s 
dissolution,  in  various  organizations  (currently,  in  the  Warfighter  Readiness  Division  [AFRL/ 
RHA];  see  Bell  and  Casey,  2007;  AFRL,  2011b).  That  work  has  included  the  development 
of  the  concept  of  Mission  Essential  Competencies,  or  MECs,  which  is  formally  defined  as  a 
“higher-order  individual,  team,  and  inter-team  competency  that  a  fully  prepared  pilot,  crew, 
flight,  operator,  or  team  requires  for  successful  mission  completion  under  adverse  conditions 
and  in  a  non-permissive  environment”  (Alliger  et  ah,  2007,  p.  14).  These  bear  similarities 
in  some  cases  to  OAD’s  task  statements,  although  they  tend  to  be  at  a  more  granular  level 
than  the  Air  Force’s  enterprise  competencies  (we  have  noted  competencies  as  a  primary  use 
in  Table  4.1).  The  methodology  behind  MECs  has  been  shared  with  sister  services,  as  well  as 
other  nations’  military  services,  including  the  United  Kingdom’s.  Currently,  AFRL/RHA  per¬ 
sonnel  are  working  with  U.S.  Army  personnel  to  apply  the  MEC  process  to  development  of 
manpower  requirements,  though  this  application  has  not,  to  date,  been  investigated  for  the  Air 
Forces’  manpower  requirement  process.  The  work  is  done  in-house,  although,  when  demand 
requires,  contractors  are  also  utilized. 

Moreover,  since  2004,  at  least  one  AFRL  711  HPW  lab  member  in  the  Human  Effective¬ 
ness  Directorate  has  been  tasked  with  selection  and  classification  support  of  AF/AIP,  applying 
data  maintained  by  SRA  for  the  purpose,  while  others  (under  the  711  HPW  suborganization 
USAFSAM)  have  conducted  selection  and  classification  work  for  high-demand  career  fields 
as  part  of  their  aerospace  medicine  consultation  mission  (noted  as  primary  uses  in  Table  4.1). 
USAFSAM  includes  at  least  one  on-site  contractor  with  a  degree  in  I/O  psychology,  although 
expertise  tends  otherwise  to  be  in  the  clinical  or  medical  context.  We  also  discovered  through 
one  of  our  other  RAND  projects  that  personnel  within  the  711th  were  in  charge  of  an  external 
contractor’s  work  assessing  the  strength  requirements  of  enlisted  AFSs  for  use  in  determining 
cut  points  on  the  Strength  Aptitude  selection  test  (noted  in  Table  4.1  as  a  secondary  use  of  JA), 
although  the  funding  for  this  may  have  been  discontinued. 

Other  Organizations 

Throughout  our  interviews,  we  identified  several  other  organizations  whose  main  mission  may 
not  be  focused  on  personnel  research  but  who  may  be  involved  in  personnel  research  in  some 
capacity.  For  example,  we  know  that  all  the  MAJCOMs  have  their  own  studies  and  analysis 
organizations  that  often  do  ad  hoc  personnel-related  research  for  the  MAJCOM.  In  addition, 
the  Air  Force  portal  had  a  survey  tool  that  could  be  used  by  any  airman  to  conduct  a  base-level 
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assessment  of  attitudes  and  opinions  about  any  topic  of  his  or  her  choosing.'^  We  also  discov¬ 
ered  that  the  Air  Force  Surgeon  General’s  Office  is  conducting  a  study  to  determine  needs  for 
job  reengineering  to  reduce  injuries  and  increase  job  safety.  Finally,  we  did  not  interview  all  the 
suborganizations  within  AFPC,  the  Air  Staff,  AETC,  or  AFRL.  Nevertheless,  we  suspect  that 
some  of  those  other  suborganizations  may  conduct  personnel-related  research.  For  example,  we 
know  AETC  often  conducts  ad  hoc  evaluations  of  education  and  training  effectiveness.  Thus, 
in  Table  4.1,  we  also  include  a  row  for  these  other  organizations  that  may  be  involved  in  per¬ 
sonnel  research  in  some  capacity.  Again,  because  we  are  unaware  of  their  exact  research  activi¬ 
ties,  we  have  included  a  “/”  to  represent  the  areas  in  which  they  may  be  involved. 

In  addition  to  these  research  efforts  conducted  in  the  Air  Eorce,  personnel  research  and 
data  collection  are  often  contracted  out  to  external  organizations.  Eor  example,  AEPC’s  SRA 
reported  that  most  of  the  studies  it  oversees  are  contracted  out.  Similarly,  external  contractors 
commonly  conducted  studies  sponsored  by  HAP,  such  as  AP/Al .  Therefore,  we  have  included 
a  “C”  in  Table  4.1  where  research  contractors  may  be  conducting  studies  for  the  Air  force. 

One  of  the  larger  external  contributors  to  Air  force  personnel  research  is  PAP,  which  has 
served  as  the  Air  force’s  only  studies  and  analysis  federally  funded  research  and  development 
center  (PPRDC)  for  more  than  60  years.  Much  of  RAND’s  PAP  work  is  directed  toward  non¬ 
personnel-related  research  areas,  such  as  acquisitions  and  logistics  or  strategy  and  doctrine. 
Nevertheless,  personnel  research  is  a  major  focus  for  one  division  within  PAP:  the  Manpower, 
Personnel,  and  Training  program.  In  this  program,  RAND’s  research  covers  a  variety  of  per¬ 
sonnel-related  issues,  including  training,  the  development  of  competencies,  and  evaluating 
current  Air  force  personnel  policies  or  systems,  following  are  some  types  of  studies  RAND 
has  conducted  for  the  Air  force  in  the  past  ten  years: 

•  assessing  training  for  cross-cultural  skills 

•  developing  officer  occupational  competencies 

•  examining  required  development  experiences  for  space  and  missile  officers 

•  examining  general  personnel  indicators  for  the  medical  and  professional  officer  corps 
(e.g.,  rates  of  accessions,  promotions,  retention) 

•  assessing  the  validity  of  the  APOQT 

•  examining  the  adequacy  of  the  current  officer  specialty  structure 

•  examining  ways  to  improve  the  development  and  utilization  of  intelligence  officers 

•  assessing  the  extent  to  which  current  pilot  training  will  meet  future  skill  requirements 

•  evaluating  reasons  for  attrition  in  certain  high- attrition  tech  training  programs 

•  identifying  the  components  of  expeditionary  predeployment  training. 

In  addition  to  RAND,  many  external  contractors  conduct  personnel  research  studies  for 
the  Air  force.  Some  of  these  organizations  are  contracted  by  the  Air  force  agencies  listed  in 
Table  4.1,  and  some  are  contracted  by  other  offices  located  throughout  the  Air  force.  We  did 
not  attempt  to  account  for  or  evaluate  the  qualifications  of  the  many  contractors  providing 
personnel  research  services  to  the  Air  force.  Nevertheless,  we  know  that,  of  the  many  contrac¬ 
tors  employed  by  the  Air  force  to  conduct  personnel  research,  some  are  reputable  personnel 
research  firms  that  have  staff  with  the  highly  specialized  skill  sets  required  to  conduct  research 
in  the  areas  listed  in  Table  4.1  (e.g.,  personnel  with  Ph.D.’s  in  psychometrics,  I/O  psychology. 


In  2011,  at  the  completion  of  this  report,  we  were  informed  that  this  tool  was  being  discontinued. 
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organizational  behavior).  We  also  know  that  it  is  highly  likely  that  some  of  the  contractors  con¬ 
ducting  this  research  for  the  Air  Force  do  not  possess  the  requisite  skills. 


Summary 

As  we  have  seen,  personnel  research  efforts  are  decentralized,  with  multiple  organizations 
involved  in  collecting  personnel-related  data  and  conducting  personnel-related  research.  These 
organizations  include  both  internal  Air  Force  organizations  and  external  contractors.  The  data 
being  collected  by  these  organizations  range  from  JA  data  to  test  scores,  performance  ratings, 
and  data  on  various  workplace  attitudes.  Research  efforts  range  from  quick  descriptive  analy¬ 
ses  of  personnel  data  to  longitudinal  studies,  such  as  those  looking  at  test  validation.  Together, 
these  organizations  are  collecting  much  of  the  key  data  required  for  current  Air  Force  needs 
and  engaging  in  important  personnel-related  research.  In  the  next  chapter,  we  address  the 
implications  of  the  current  structure  and  whether  the  myriad  of  data  collected  by  the  many 
organizations  is  sufficient  to  meet  the  needs  of  the  Air  Force. 


CHAPTER  FIVE 


An  Examination  of  How  Well  the  Current  Structure  of  Personnel- 
Related  Research  Efforts  Meets  Air  Force  Needs 


The  main  study  purpose  was  to  assess  how  well  the  current  organizational  structure  of  person¬ 
nel  research  activities  can  meet  the  needs  of  the  broader  Air  Force.  In  this  chapter,  we  seek  to 
answer  this  question. 

The  Air  Force  is  conducting  a  great  deal  of  personnel-related  research,  despite  the  cur¬ 
rent  decentralized  nature  of  the  organizational  structure.  The  various  organizations  involved 
in  personnel  research  engage  in  a  wide  variety  of  investigations  and,  taken  together,  collect  all 
three  types  of  data  typically  used  in  personnel  research  (i.e.,  administrative  data,  attitude  and 
perception  data,  and  JA  data)  that  form  the  cornerstone  of  data-driven  HRM  decisionmaking. 
However,  we  found  that  there  are  also  some  critical  issues  that  currently  inhibit  the  quality  and 
efficiency  of  current  personnel  research  efforts,  including  the  following: 

•  narrow  organizational  missions 

•  inconsistent  data-collection  coordination  or  data  sharing 

•  a  lack  of  internal  personnel  research  expertise 

•  limited  resources 

•  reliance  on  contractors 

•  potential  duplication  of  effort. 

Table  5.1  provides  an  overview  of  which  issues  were  identified  (indicated  by  a  check¬ 
mark)  in  the  organizations  we  reviewed  in  Chapter  Four.  In  cases  in  which  we  did  not  have 
enough  information  to  identify  whether  an  issue  applied,  we  have  included  a  “/”  in  the  column. 
Similarly,  when  an  issue  was  not  applicable  to  the  activities  or  mission  of  an  organization,  we 
noted  this  by  including  “NA”  in  the  column.  The  intersection  of  narrow  missions,  inconsistent 
data  collection  and  sharing,  and  inconsistent  communication  leads  to  potential  duplication  of 
effort;  we  do  not  include  that  as  a  separate  column  in  Table  5.1  specifically,  though  potential 
areas  of  overlap  include  work  on  competencies  and  selection  (the  reader  is  referred  to  Table  4.1 
in  Chapter  Four  to  see  some  additional  potential  areas).  We  then  discuss  each  of  the  issues  we 
identified  in  greater  depth.  Although  these  issues  rest  on  information  gathered  in  this  study, 
they  are  similar  to  issues  seen  by  other  observers  of  government,  including  the  U.S.  Govern¬ 
ment  Accountability  Office  (GAO)  (e.g.,  GAO  [then,  the  U.S.  General  Accounting  Office], 
2002)  and  Liebowitz  (2004). 
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Table  5.1 

Critical  Issues  Regarding  Current  Personnel  Research  Efforts 


Organization 

Narrow  Mission 

Inconsistent 
Data-Coilection 
Coordination  or 
Sharing 

Lack  of  Personnei 
Research 
Expertise 

Limited  Resources 

Reiiance  on 
Contractors 

A  ETC 

OAD 

✓ 

✓ 

✓ 

✓ 

AAD 

✓ 

NA 

✓ 

SAS 

/ 

✓ 

✓ 

✓ 

AFPC 

DSYD 

✓ 

NA 

DSYA 

/ 

✓ 

SRA 

✓ 

✓ 

✓ 

AFMA 

MAS 

✓ 

✓ 

MAP 

✓ 

/ 

✓ 

✓ 

HQ  AF/A1 

AF/A1D 

✓ 

NA 

NA 

✓ 

AF/A1P 

✓ 

NA 

NA 

✓ 

AF/A1S 

✓ 

NA 

NA 

✓ 

AIM 

/ 

NA 

NA 

/ 

HQ  AF/A9 

/ 

NA 

NA 

/ 

AFRL711  HPW 

711  HPW/RH 

✓ 

✓ 

✓ 

USAFSAM 

✓ 

✓ 

Narrow  Organizational  Missions 

The  first  issue  we  identified  affecting  current  personnel  research  efforts  within  the  Air  Force 
is  that  several  of  the  organizations  have  very  specific,  narrow  missions  focused  on  support¬ 
ing  the  needs  of  their  larger  parent  organizations.  As  Table  5.1  shows,  this  was  an  issue  for 
OAD,  AAD,  AFPC’s  DSYD,  and  both  of  AFMA’s  divisions  involved  in  personnel  research. 
Specifically,  OAD  described  its  main  focus  as  collecting  OA  data.  AAD  described  a  sole  focus 
on  developing  study  guides  and  tests  in  support  of  WAPS.  AFPC’s  DSYD  maintains  broad 
personnel  databases  but  described  its  primary  purpose  as  developing  summary  reports  of  the 
personnel  data  only.  Finally,  of  AFMA’s  two  divisions,  MAS  described  its  mission  as  focused 
on  developing  manpower  standards,  while  MAP  described  its  main  purpose  as  hosting  and 
providing  oversight  of  all  Air  Force-wide  attitude  and  opinion  surveys.  In  contrast,  the  other 
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organizations  we  examined  described  themselves  as  having  much  broader  missions  instead  of 
this  more  singular  focus. 

Having  a  narrow  organizational  mission  is  not  necessarily  an  impediment  to  conduct¬ 
ing  quality  personnel  research,  but,  in  certain  cases,  it  can  inhibit  the  ability  to  meet  the  Air 
Force’s  broader  needs.  For  example,  OAD  described  its  main  mission  as  developing  JI  surveys 
to  collect  data  on  the  tasks  people  do  for  enlisted  occupations.  The  JA  data  it  collects  are  then 
used  as  input  for  developing  and  validating  training,  for  job  classification,  and  for  supporting 
the  development  of  promotion  tests  and  study  guides  in  AAD.  When  asked  to  describe  other 
potential  uses  it  saw  for  the  data  collected,  OAD  stated  that  it  had  not  thought  about  how 
the  data  could  be  used  in  other  ways.  Instead,  it  was  focused  on  fulfilling  its  primary  mission 
and  supporting  the  needs  of  AETC,  its  parent  organization.  However,  as  discussed  in  Chapter 
Two,  JA  data  serve  as  the  foundation  for  much  of  personnel  research  and  can  be  used  across  the 
personnel  spectrum,  such  as  in  recruiting,  selection,  and  performance  appraisal.  As  an  example 
of  such  broader  applications,  before  the  disestablishment  of  AFHRL  and  its  heirs,  work  was 
being  done  on  an  application  of  OA  to  a  performance  management  system  (see  Hedge  and 
Teachout,  1992).  Although  OAD  reports  presently  being  engaged  in  broader  marketing  and 
collaborative  efforts,  the  JA  data  collected  by  OAD  are  not  used  in  this  broader  manner.  Other 
organizations’  lack  of  awareness  regarding  availability  and  usefulness  of  these  data  was  con¬ 
firmed  in  our  interviews  with  the  various  organizations  we  identified  as  involved  in  personnel 
research.  Thus,  in  cases  such  as  this,  the  narrow  mission  of  the  organization  actually  results  in 
data  not  being  used  to  the  fullest  extent  and  inhibits  the  ability  of  personnel  research  to  meet 
the  needs  of  the  broader  Air  Force. 


Inconsistent  Data-Collection  Coordination  or  Data  Sharing 

A  second  issue  we  identified  was  that  there  was  often  inconsistent  coordination  of  data- 
collection  efforts  and  inconsistent  data  sharing.  As  Table  5.1  shows,  this  was  an  issue  for  OAD; 
SAS;  AFPC’s  SRA;  AFMA’s  MAS,  AF/AID,  AF/AIP,  and  AF/AIS;  and  711  HPW/RH  and 
USAFSAM. 

In  many  cases,  a  lack  of  data- collection  coordination  was  related  to  the  issue  of  organiza¬ 
tions  focused  on  fulfilling  very  specific,  narrow  organizational  missions.  For  example,  OAD 
and  AFMA’s  MAS  both  collect  JA  data,  but  they  do  not  currently  coordinate  any  of  their  data- 
collection  efforts.  OAD  is  the  designated  organization  for  OA  (JA)  in  the  Air  Force  and  uses 
a  JI  survey  to  collect  information  on  the  tasks  people  perform,  the  time  they  spend  on  each 
task  relative  to  other  tasks,  and  the  difficulty  of  those  tasks.  To  develop  manpower  standards, 
AFMA’s  MAS  needs  data  on  the  amount  of  time  people  spend  on  each  task.  However,  this  is 
not  something  that  OAD’s  JI  survey  currently  collects.  Therefore,  AFMA  independently  col¬ 
lects  these  additional  data  on  the  tasks  people  perform  in  their  occupations.  Thus,  although 
these  two  organizations  are  both  collecting  similar  data,  and  it  would  seem  relatively  easy  for 
this  to  be  added  to  OAD’s  survey,  their  data  collections  are  not  currently  coordinated.  This 
appears  to  be  partly  the  result  of  their  focuses  on  different  missions  and  purposes  for  the  data. 

Similarly,  other  organizations  view  the  data  they  collect  as  serving  their  own  very  spe¬ 
cific  research  purposes,  resulting  in  inconsistent  coordination  and  sharing  with  others.  For 
example,  although  SAS  does  not  necessarily  have  a  narrow  organizational  mission  in  terms  of 
the  personnel  research  it  conducts,  it  described  its  research  as  focused  on  supporting  its  parent 
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organization,  AETC.  Therefore,  much  of  the  data  it  collects  are  not  shared  and  made  known  to 
the  wider  Air  Force  community  because  the  data  are  collected  for  research  focused  on  training 
and  education.  Likewise,  AF/AID  and  AF/AIP  both  described  conducting  research  to  exam¬ 
ine  how  to  apply  competencies  to  the  personnel  management  system.  However,  they  each  view 
their  own  research  on  competencies  as  very  distinct  from  the  others  and  reported  seeing  little 
overlap  in  their  efforts.  Therefore,  although  they  might  benefit  from  coordination  and  data 
sharing,  they  reported  not  doing  so.  The  711  HPW/RH  also  conducts  research  on  (mission- 
essential)  competencies;  however,  they  noted  that  their  work  is  actually  more  similar  to  the 
work  conducted  by  OAD  in  terms  of  granularity  and  hence  coordination  and  data  sharing 
with  OAD  would  have  more  utility.  This  coordination  has  not  been  pursued  actively  because 
of  time  and  resource  constraints. 

Finally,  USAFSAM  also  examines  issues  related  to  selection  testing  but  views  its  work 
as  very  occupationally  specific,  in  contrast  to  AlP’s  Air  Force-wide  mandate;  thus,  in  this 
instance,  data  sharing  and  coordination  are  not  considered  relevant,  although  presumably 
findings  from  both  general  and  specific  cases  have  potential  to  cross-generalize. 

In  other  cases,  there  is  inconsistent  data  sharing  because  the  data  may  be  housed  with 
contractors  instead  of  the  sponsoring  organization.  For  example,  the  CA  conducted  by  AF/ 
AIS  was  administered  online  through  a  contractor  because  AFMA  did  not  have  the  capacity 
to  host  the  survey  or  store  data.  The  MAJCOM  and  Air  Staff  own  the  database,  but  it  is  now 
stored  with  the  external  contractor.  As  a  result,  any  historical  data  analysis  needed  (e.g.,  differ¬ 
ent  displays  of  results)  requires  specific  separate  funding  for  the  contractors  to  pull  and  present 
the  data.  Thus,  the  availability  of  the  data  to  other  Air  Force  organizations  is  limited. 

Finally,  inconsistent  data  collection  and  sharing  also  occur  because  of  a  lack  of  awareness 
about  other  organizations  that  may  be  involved  in  similar  personnel  data- collection  activi¬ 
ties  or  research.  For  example,  AFPC’s  SRA  is  in  the  process  of  updating  and  maintaining  an 
HRRD,  which  is  designed  to  maintain  an  archive  of  human  resource  data  from  the  past  50 
years.  However,  this  branch  reported  that  most  of  the  other  Air  Force  organizations  are  not 
aware  of  the  work  it  does  and,  therefore,  are  less  likely  to  take  advantage  of  the  data  it  collects 
and  stores. 


A  Lack  of  Internal  Personnel  Research  Expertise 

A  third  issue  we  identified  is  that  several  of  the  organizations  we  interviewed  lack  the  neces¬ 
sary  internal  personnel  research  expertise  to  effectively  carry  out  their  activities  to  the  highest- 
quality  standards  possible.  As  Table  5.1  shows,  this  was  particularly  an  issue  for  OAD,  AAD, 
and  SAS. 

For  example,  SAS  receives  various  requests  from  other  Air  Force  organizations  to  conduct 
studies,  but  it  reported  a  limited  capacity  to  meet  many  of  these  requests.  Although  it  has  indi¬ 
viduals  on  staff  with  research  backgrounds  and  Ph.D.’s,  most  of  its  expertise  is  in  operations 
research,  which  does  not  necessarily  fit  the  skill  sets  needed  for  many  of  the  study  requests.  In 
some  cases,  it  has  solicited  the  expertise  of  others  outside  of  SAS,  such  as  professors  at  the  Air 
University  or  personnel  in  AFPC’s  SRA.  However,  it  often  has  to  turn  requests  away  that  it 
does  not  feel  that  it  has  the  expertise  to  meet,  especially  when  these  studies  fall  outside  of  its 
main  mission. 
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In  addition,  as  we  learned  in  our  interviews  with  representatives  from  OAD  and  AAD, 
these  organizations  have  continued  using  largely  the  same  methods  for  WAPS  test  develop¬ 
ment  and  OA  data  collection  and  analysis  that  were  established  hy  AFHRL  many  years  ago, 
though  the  organizations  have  made  improvements  in  delivery  (i.e.,  going  to  weh-hased  ser¬ 
vices),  tracking,  and  software.  AFHRL  was  supporting  continued  development  of  OA  soft¬ 
ware  and  of  promotion  testing  techniques  as  recently  as  1989  (Buescher,  Olvera,  and  Besetsny, 
1989).  Since  that  time,  however,  progress  in  the  field  of  personnel  research  has  been  made  that 
would  potentially  benefit  AAD  and  OAD. 

We  found  that  AAD  does  not  have  the  necessary  internal  expertise  to  do  its  own  psycho¬ 
metric  analyses.  Currently,  AAD  conducts  basic  descriptive  analyses  on  its  promotion  tests  by 
using  an  automated  computer  program  developed  years  ago.  Although  this  program  provides 
many  of  the  necessary  basic  descriptive  statistics,  it  is  unclear  how  some  of  the  statistics  are  cal¬ 
culated.  In  addition,  there  has  been  advancement  in  such  areas  as  differential  item  functioning 
(using  item  response  theory  rather  than  classical  test  theory)  that  could  benefit  AAD.  However, 
to  our  knowledge,  it  does  not  currently  have  the  internal  expertise  to  address  these  issues  and 
could,  therefore,  benefit  from  adding  Ph.D.’s  with  personnel  or  testing  research  backgrounds, 
such  as  psychometrics,  education,  or  I/O  psychology. 

Finally,  OAD  reported  having  the  expertise  needed  for  its  current  work,  which  is  primar¬ 
ily  descriptive,  but  reported  that  it  did  not  have  the  necessary  expertise  to  do  more-advanced 
statistical  analyses  for  its  own  work  or  what  is  required  for  some  of  the  ad  hoc  study  requests 
it  receives. 


Limited  Resources 

In  addition  to  a  lack  of  internal  research  expertise,  several  organizations  also  reported  having 
limited  resources,  in  terms  of  both  funding  and  available  staff.  As  Table  5.1  shows,  this  is  an 
issue  for  OAD,  SAS,  AFPC’s  DSYA  and  SRA,  AFMAs  MAP,  and  the  711  HPW/RH. 

Most  of  these  organizations  reported  that  they  had  enough  resources  to  conduct  their 
current  activities.  However,  they  reported  that  they  often  do  not  have  the  necessary  funding 
or  staff  numbers  to  be  able  to  respond  to  many  external  study  requests.  For  example,  AFMAs 
MAP  and  its  subordinate  unit,  the  Air  Force  Survey  Office,  is  the  designated  authority  for 
reviewing  and  approving  any  Air  Force-wide  opinion-  and  attitude-based  surveys.  As  a  result, 
many  organizations  also  solicit  its  expertise  in  survey  development  and  analysis.  However, 
MAP  reported  a  limited  capacity  to  help  those  organizations  desiring  this  kind  of  expertise 
because  of  limited  staff  and  funding.  Additionally,  their  limited  resources  can  also  lead  to  sig¬ 
nificant  waiting  periods  for  survey  approvals.  The  711  HPW  personnel  with  whom  we  spoke 
also  indicated  that  the  resources  to  support  general  personnel  research  were  difficult  to  secure 
and  that  repeated  attempts  to  do  so  had  not  succeeded. 

Though  such  resource  challenges  may  be  common  in  today’s  Air  Force  and  indeed  in 
many  organizations,  the  lack  of  resources  supporting  a  broader  personnel  research  agenda  in 
some  fashion  is  nonetheless  problematic.  A  time  of  tightening  budgets  suggests  that  thoughtful 
force  restructuring  from  a  strategic  research  perspective  would  be  a  mandate;  however,  the  cur¬ 
rent  resource  situation  among  organizations  conducting  personnel  research  is  too  constricting 
to  even  support  requests  outside  individual  organizations’  immediate  missions.  Certainly,  no 
organization  currently  has  the  explicit  broad  bandwidth  and  resources  to  engage  in  such  per- 
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sonnel  research  in  a  strategic,  forward-thinking  rather  than  reactive,  manner.  Even  AFRL’s  711 
HPW,  armed  with  its  research  and  development  mandate,  is  unahle  to  undertake  the  effort  on 
the  hroad  scale  that  would  he  required  to  meet  the  needs  of  the  Air  Force  as  a  whole. 


Reliance  on  Contractors 

Because  of  a  lack  of  internal  research  expertise  and  funding  for  same,  several  organizations  also 
reported  relying  on  contractors  to  conduct  much  of  their  research.  As  Table  5.1  shows,  this  is 
an  issue  for  AFPC’s  SRA,  AFMAs  MAP,  AF/AID,  AF/AIP,  AF/AIS,  the  711  HPW/RH,  and 
USAFSAM. 

Although  contractors  can  serve  a  useful  function,  a  heavy  reliance  on  them  can  also 
result  in  several  issues  when  it  comes  to  maintaining  institutional  knowledge  and  making 
data  available  to  the  wider  Air  Force.  For  example,  in  cases  in  which  different  contractors  are 
involved  in  personnel  research,  this  dispersion  of  work  makes  it  more  difficult  to  maintain 
internal  Air  Force  knowledge  of  what  data  were  collected,  what  studies  were  conducted,  and 
what  key  findings  were  obtained  in  past  years.  As  already  mentioned,  the  data  may  also  reside 
with  contractors,  potentially  limiting  their  wider  use  by  other  Air  Force  organizations.  Also, 
relying  on  contractors  can  be  problematic  when  the  individuals  making  the  decisions  about 
the  appropriate  contractor  to  use  for  a  project  have  limited  knowledge  about  the  capabilities  of 
the  contractor.  This  can  result  in  contractors  being  hired  that  do  not  have  the  necessary  exper¬ 
tise  or  knowledge  of  the  Air  Force  (or,  unfortunately,  necessary  expertise  or  knowledge  more 
generally).  Thus,  with  no  broad  oversight  of  the  various  external  contractors  collecting  data 
or  what  happens  to  the  data  following  the  study,  a  reliance  on  contractors  can  create  further 
challenges.  Contractors  can  supply  a  wide  variety  of  focused  expertise  that  would  otherwise  be 
beyond  the  funding  capability  of  the  government  and  hence  do  provide  efficiencies;  moreover, 
in  some  cases,  they  help  satisfy  demand  that  temporarily  outstrips  government  capacity.  How¬ 
ever,  insufficient  in-house  expertise  for  the  Air  Force  to  leverage  in  contractor  selection  may 
preclude  capitalizing  on  this  efficiency. 


Potential  Duplication  of  Effort 

Finally,  because  of  both  a  lack  of  communication  and  data  sharing  among  these  organizations 
and  a  lack  of  an  overall  personnel  research  alignment,  there  is  the  potential  for  duplication  of 
effort.  Without  a  single  entity  in  charge  of  personnel  research,  consumers  of  that  research  are 
left  with  no  guidance  about  whom  to  contact  to  get  that  research  accomplished,  and  several 
Air  Force  organizations  accept  requests  to  conduct  personnel  research  or  engage  in  research  to 
support  their  own  ends  without  consideration  of  cross-Air  Force  similarities  in  data  needs  or 
strategic  future  data  applications. 

As  shown  in  Table  4.1  in  Chapter  Four,  organizations  are  involved  in  collecting  similar 
personnel  data  and  conducting  research  on  related  topics.  For  example,  there  is  an  overlap  in 
collecting  JA-type  data.  OAD  is  the  designated  organization  for  OA  (JA)  in  the  Air  Force. 
However,  other  organizations,  including  AFMAs  MAS,  AFPC’s  SRA,  several  AF/Al  divi¬ 
sions,  and  the  711  HPW/RH,  also  reported  collecting  JA-type  data.  Currently,  however,  these 
organizations  do  not  report  coordinating  these  efforts.  As  noted,  the  711  HPW/RH’s  work  on 
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MECs  is  perceived  as  more  similar  to  OAD’s  JA  work  (rather  than  competencies  per  se);  in 
that  instance,  there  has  been  some  communication  between  the  two  organizations  regarding 
the  similarities,  but  time  and  resource  constraints  have  not  permitted  a  systematic  exploration. 
Coordination  with  other  organizations  doing  work  on  competencies  is  not  seen  as  applicable. 

Another  area  of  potential  duplication  of  effort  is  research  on  competencies.  Both  AF/AID 
and  AF/AIP  reported  conducting  research  on  how  to  apply  competencies  to  the  personnel 
management  system.  However,  they  did  not  report  coordinating  any  of  their  current  efforts 
and  saw  their  research  on  competencies  as  very  distinct  given  AF/AlD’s  focus  on  enterprise 
competencies  and  AF/AlP’s  focus  on  occupational  competencies.  Given  that  enterprise  com¬ 
petencies,  by  design,  extend  across  the  Air  Force  and  are  present  in  all  jobs  (albeit  in  varying 
levels)  and  given  the  likely  (and  admitted,  as  in  job  groupings)  commonality  across  occupations 
in  the  Air  Force,  these  enforced  distinctions  seem  unwarranted. 

Thus,  our  research  revealed  that  several  organizations  are  involved  in  similar  data- 
collection  efforts  and  in  conducting  research  on  similar  personnel  topics.  However,  with  no 
centralized  body  overseeing  this  type  of  work,  these  organizations  are  often  unaware  of  the 
similar  efforts  or,  even  when  aware  of  them,  may  not  see  the  similarities.  As  a  result,  there  is 
the  potential  for  unnecessary  duplication  of  effort  that  costs  the  Air  Force  money  and  reduces 
the  efficiency  of  personnel  research  efforts  overall. 
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Personnel  research  is  the  bedrock  of  a  well-designed  personnel  management  system.  It  serves 
as  a  scientific  basis  for  identifying,  diagnosing,  and  solving  problems  in  the  personnel  system 
and  offers  data-driven  evidence  about  the  impact  of  changes  to  the  personnel  system.  However, 
the  impact  of  personnel  research  can  be  easily  overlooked.  Personnel  management  systems  usu¬ 
ally  continue  to  function  without  it;  decisions  about  whom  to  hire,  whom  to  fire,  what  type 
of  training  is  needed,  and  other  personnel  matters  continue  to  be  made  in  its  absence.  And 
many  organizations  continue  to  exist  and  even  thrive  without  it.  So  why  should  the  Air  Force 
be  concerned  about  personnel  research? 

Although  many  of  the  key  elements  of  a  smoothly  working  personnel  research  system 
exist  in  the  Air  Force  or  can  be  brought  to  bear  with  help  from  outside  contractors,  the  current 
system  is  not  ideal  in  terms  of  the  Air  Force’s  personnel  needs.  Although  the  current  system 
could  probably  continue  and  maintain  the  existing  overall  personnel  system  at  less-than-opti- 
mal  efficiency  for  some  time,  action  now  would  be  helpful  to  forestall  systemic  issues  and  align 
the  system.  Although  our  approach  did  not  permit  us  to  determine  specific  estimates  of  the 
cost  of  lost  efficiency  in  the  personnel  research  system  that  supports  personnel  policy  decisions, 
the  high  cost  of  personnel  as  a  whole  (approximately  $37  billion  in  the  active  Air  Force  alone 
for  2013;  DoD,  2011)  suggests  that  even  minuscule  gains  have  potential  for  large  cost  savings, 
while  minuscule  inefficiencies  can  impose  large  costs. 

The  Air  Force  is  entering  a  time  of  resource  constraints  and  will  be  making  changes  to 
the  personnel  system  in  consequence.  Restructuring  and  downsizing  should  be  informed  by 
research  because  such  changes  may  be  costly.  Certainly,  the  literature  on  organizational  down¬ 
sizing  and  mergers  and  acquisitions  suggests  that  the  costs  of  organizational  changes  can  be 
high  in  terms  of  employee  psychological,  physical,  and  economic  health;  findings  apply  both 
for  those  who  remain,  as  well  as  those  who  leave  the  organization.  The  literature  from  the 
macro  perspective  is  equivocal  at  best  in  terms  of  gains  in  organizational  profitability  and  other 
strategic  outcomes,  so  a  holistic  perspective  suggests  that  downsizing  requires  a  considered 
approach  (see,  e.g.,  the  recent  review  by  De  Meuse,  Marks,  and  Dai,  2011). 

Finally,  a  quantitative  summary  of  the  strategic  human  resource  literature  suggests  that 
systems  of  personnel  practices  have  a  positive  effect  on  various  measures  of  organizational  per¬ 
formance,  including  accounting  returns,  market  returns,  and  retention,  and  that  systems  of 
practices  have  a  stronger  positive  effect  than  do  individual  best  practices  (Combs  et  ah,  2006). 
Without  consideration  of  the  contingencies  that  are  being  reinforced  throughout  the  system 
(and  a  true  perception  of  the  contingencies  at  the  individual  level  obtained  through  asking 
employees),  disorganized  personnel  practice  and  policy  strategies  can  result  in  reinforcement  of 
behaviors  that  contravene  the  interests  of  both  the  organization  and  potentially  even  individual 
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employees.  For  example,  the  Air  Force  is  both  a  military  organization  and  a  nuclear  enterprise; 
thus,  the  enforcement  of  safety  behaviors  and  maintenance  of  a  high  safety  climate  are  key  to 
success.  Deviations  from  safety  climate  and  policies  that  support  such  climate  can  be  informed 
by  research  (e.g.,  Clarke,  2006).  Thus,  organizational  promotion  policies  and  practices  that 
reward  “meeting  mission”  more  rapidly  in  a  time  of  war,  without  due  process  for  safety  require¬ 
ments,  might  be  one  example  of  a  performance  management  system  that  does  not  align  with 
the  interests  of  the  organization.'  Of  course,  such  failures  of  safety  climate  can  result  in  conse¬ 
quences  negative  for  both  the  Air  Force  and  its  employees. 

To  summarize,  the  personnel  system  is  important,  and  better  alignment  (and  sufficient 
research  to  support  that  alignment)  can  yield  dividends.  The  Air  Force  should  care  about  that 
alignment  and  support  it  through  research  that  is  similarly  aligned  to  meet  mission.  Although 
personnel  research  in  a  strategic  sense  has  not  been  an  Air  Force  priority  for  many  years  (as 
evidenced  by  dearth  of  funding),  a  strategic  approach  now  would  be  a  worthy  priority  for  Air 
Force  leadership  and  is  one  being  undertaken  by  the  federal  government  more  widely.  GAO 
(2002)  explicitly  states  in  its  model  for  human  capital  management  that  two  critical  success 
factors  for  strategic  human  capital  planning  are  data-driven  decisions  and  alignment  with  the 
organization’s  strategic  needs.  Success  is  measured  by  how  well  strategic  objectives  are  met;  for 
the  personnel  system  of  the  Air  Force,  this  means  that  success  can  be  measured  by  how  well  the 
Air  Force  maintains,  retains,  and  develops  a  workforce  with  the  requisite  KSAOs  that  enables 
the  service  to  fly,  fight,  and  win;  enables  a  climate  that  facilitates  this  process;  and  maintains 
the  job  satisfaction  of  that  workforce  (given  job  satisfaction’s  established  links  to  retention).  A 
prospective  rather  than  reactive  perspective  inheres  in  the  consideration  of  strategic  outcomes, 
especially  in  times  of  resource  constraints,  when  a  long-term  outlook  is  necessary  to  determine 
reasonable  trade-offs  between  short-term  and  long-term  objectives.  This  success  requires  data 
to  inform  policy  decisionmaking  about  force  shaping.  Liebowitz  (2004)  notes  that,  in  their 
extensive  writings  on  the  topic  for  the  context  of  the  government  workforce,  both  the  Office 
of  Personnel  Management  (OPM)  and  GAO  have  emphasized  the  need  for  organizations  “to 
become  less  hierarchical,  process-oriented,  ‘stovepiped,’  and  inwardly  focused”  (p.  8). 

In  this  chapter,  we  make  some  recommendations  about  what  components  should  be  con¬ 
sidered  in  undertaking  change  and,  ultimately,  provide  some  alternative  organizational  struc¬ 
tures  as  a  starting  point  for  discussion.  The  issues  we  identified  in  the  current  personnel  system 
should  be  alleviated,  with  an  integrated  and  forward-looking  system  set  up  to  enable  their  reso¬ 
lution  and  provide  better  capacity  for  research  to  support  essential  personnel  policy  decisions. 
In  some  cases,  personnel  research  can  itself  inform  some  of  the  considerations  to  implement 
these  components  appropriately. 


Essential  Components  for  Change 

Any  change  effort  undertaken  by  the  Air  Force  should  contain  specific  elements  that  will 
enable  the  personnel  system  to  meet  the  challenges  of  the  future.  These  components  flow 
directly  from  the  overall  research  findings  of  this  study  and  seek  to  resolve  the  existing  issues. 


'  A  performance  management  system  consists  of  the  specific  personnel  policies  and  procedures  for  evaluating  and  docu- 
menting  performance.  Personnel  management  is  the  larger  overall  structure  within  which  a  performance  management 
system  exists. 
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although  they  may  he  operationalized  in  a  variety  of  different  ways.  We  identified  the  critical 
gaps  in  the  Air  Force’s  current  personnel  research  system  of  narrow  missions,  inconsistent  data 
collection  and  coordination,  lack  of  requisite  expertise,  limited  resources,  reliance  on  contrac¬ 
tors,  and  potential  duplication  of  effort.  Thus,  the  change  components  we  recommend  seek  to 
address  these. 

Oversight  over  All  Personnel  Research  Efforts 

One  component  that  is  clearly  missing  in  the  current  system  is  an  organizational  structure 
that  has  clear  oversight  responsibility  over  all  the  personnel  research  efforts  ongoing  in  the  Air 
Force.  Although  AFMA  is  the  huh  for  Air  Force-wide  attitude  surveys  and  other  organizations 
serve  as  the  hubs  for  various  other  types  of  research  (e.g.,  AF/AIP  serves  as  the  hub  for  selec¬ 
tion  and  testing  research),  no  one  organization  or  structure  actively  coordinates  them  all.  To 
make  sure  the  entirety  of  collected  data  is  brought  to  bear  on  personnel  decisionmaking  and 
to  perceive  where  gaps  in  the  personnel  data  available  exist  that  may  result  in  poor  personnel 
decisionmaking,  some  oversight  is  needed.  This  organizational  structure  should  be  informed 
in  some  fashion  of  all  work  being  done — attitudinal  or  otherwise,  contractor  or  otherwise, 
operational  or  otherwise — so  that  its  usefulness  for  the  system  may  be  judged  and  properly 
integrated.  This  oversight  would  enable  the  alleviation  of  problems  occasioned  by  narrow  orga¬ 
nizational  missions  and  enable  awareness  of  inconsistent  data-sharing  and  duplication  issues  in 
order  to  resolve  them. 

Oversight  is  more  than  just  awareness,  however.  The  relative  stability  of  AFOMS  meth¬ 
odology  over  the  decades  as  technologies  and  best  practices  updated  does  suggest  that  the 
mere  existence  of  an  organization,  such  as  AFHRL,  with  the  explicit  mission  set  for  personnel 
research  and  development  is  insufficient  to  overcome  funding  and  organizational  stovepiping 
issues:  Oversight  responsibility  must  be  explicit  to  allow  expertise  to  inform  operational  pro¬ 
cesses.  Moreover,  this  oversight  should  be  married  with  an  explicit  mandate  to  consider  long¬ 
term  Air  Force  needs  and  engage  in  research  and  development.  The  longer  view  is  essential  in 
an  organization  with  restrictions  on  lateral  hiring  and  the  subsequent  need  to  ensure  that  the 
fruits  of  recruitment  and  selection  pay  off  20  and  30  years  down  the  line,  through  the  course 
of  careers,  in  mature  officers  and  NCOs  able  to  guide  the  Air  Force  itself  This  helps  answer  the 
call  to  shift  to  a  strategic  approach  to  human  capital  management,  as  issued  by  GAO  (Walker, 
2002). 

Sufficient  Authority  to  Coordinate  Efforts 

To  achieve  the  benefits  of  oversight,  it  is  necessary  for  the  organizational  structure  chosen 
to  have  sufficient  authority  to  coordinate  the  disparate  elements  of  the  Air  Force’s  existing 
personnel  research  system.  Knowing  what  data  are  available  to  Air  Force  decisionmakers  is 
one  step;  the  next  step  is  having  the  authority  to  execute  actions  to  eliminate  duplication  of 
effort  for  greatest  efficiency,  require  the  disparate  elements  to  communicate  and  share  data  as 
needed,  and  institute  new  research  or  collection  of  new  data  elements  where  gaps  in  the  req¬ 
uisite  data  currently  exist.  Such  gaps  include  questions  that  may  be  anticipated  strategically 
as  the  Air  Force  restructures  to  make  best  use  of  its  resources.  The  structure  must  also  have 
sufficient  authority  and  legitimacy  to  request  the  existing  data  from  the  collecting  organiza¬ 
tions  to  inform  any  decisionmaking  for  the  personnel  system  as  a  whole  and  to  require  exist¬ 
ing  organizations  to  restructure  their  processes  when  necessitated  by  changes  in  best  practices. 
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A  performance  management  system  that  reinforces  cooperation  and  coordination  would  help 
enable  this  authority,  as  would  a  concerted  effort  at  support  from  Air  Force  leadership. 

Institutional  Knowledge 

Ideally,  this  organizational  structure  will  also  include  the  component  of  institutional  knowl¬ 
edge.  A  deep  understanding  of  the  Air  Force  and  the  way  things  are  customarily  done — its 
culture,  values,  and,  of  course,  its  personnel  system — will  assist  in  determining  true  gaps  in 
knowledge  and  the  collection  of  the  most-ideal  data  to  fill  these  gaps.  Data  collection  is  needed 
for  organizational  decisions,  hut,  if  the  wrong  questions  are  asked  because  of  a  misunderstand¬ 
ing  about  the  Air  Force,  the  data  are  meaningless  and  the  effort  to  collect  them  wasteful.  In 
addition,  having  an  intimate  understanding  of  the  Air  Force  as  an  organization  furthers  an 
understanding  of  what  information  is  available.  Historical  understanding  and  institutional 
knowledge  will  also  inform  efforts  to  coordinate  disparate  organizational  elements  as  needed, 
enable  the  organizational  structure  to  leverage  authority  more  effectively,  and  understand  what 
resources  might  best  be  brought  to  bear  to  tackle  a  given  personnel  system  problem. 

Quality  Control 

The  component  of  institutional  knowledge  illuminates  the  need  for  an  additional  component: 
quality  control.  Currently,  even  as  no  one  organization  is  responsible  for  oversight  of  personnel 
research  efforts  overall,  no  one  organization  is  responsible  for  ensuring  that  ongoing  and  one- 
off  research  efforts  meet  minimum  standards  for  quality  and  utility.  In  some  cases,  individual 
organizations  may  not  have  the  requisite  knowledge  even  to  choose  an  appropriately  equipped 
contractor.  Institutional  knowledge  can  help  ensure  that  research  efforts  collect  the  correct 
information  to  answer  the  question  under  investigation.  A  quality  control  function  ensures 
that  collected  data  can  be  more  easily  integrated  into  a  smoothly  functioning  system  and  that 
a  question  does  not  remain  unanswered  because  data  collected  for  one  or  more  aspects  of  the 
problem  are  inadequate  to  meet  the  need.  Quality  control  is  necessary,  in  short,  to  ensure  that 
the  data  collected  are  appropriate  to  the  purpose  and  serve  the  needs  of  the  Air  Force  with 
efficiency. 

Access  to  Scientific  Expertise  and  Resources 

Without  expertise  and  other  resources,  quality  control  is  just  a  compelling  slogan  rather  than  an 
operationalized  component.  Perception  of  the  ideal  integration  of  different  aspects  of  person¬ 
nel  data  collected  to  answer  a  given  research  question  and  the  ability  to  monitor  the  rigor  with 
which  the  data  are  collected  and  determine  whether  quality  is  adequate  require  a  background 
in  research  design.  Methodological  expertise  is  not  enough,  however:  Appropriate  theoretical 
orientations  will  be  required  as  well  to  enable  a  more  strategic,  long-term,  and  aligned  view. 
This  suggests  that  appropriate  backgrounds  include  I/O  psychology,  organizational  behavior, 
or  related  fields.  Multiple  perspectives  and  research  backgrounds  would  probably  be  useful. 
Personnel  with  such  backgrounds  would  be  helpful  in  filling  any  gaps  in  expertise  at  various 
existing  Air  Force  organizations,  such  as  those  we  identified.  Further  appropriate  expertise 
would  assist  planning  efforts  to  enable  existing  structures  to  reorganize  data- collection  efforts 
for  greatest  efficiency  and,  when  needed,  determine  appropriate  contracting  support.  Person¬ 
nel  are  expensive.  Data  collection  can  also  be  expensive.  Thus,  a  key  aspect  of  this  component 
would  also  be  sufficient  resources  in  terms  of  funding,  as  well  as  expertise,  to  enable  the  Air 
Force  to  optimize  its  personnel  system  and  operate  in  the  most  efficient  manner  possible  with- 
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out  overburdening  existing  or  new  organizational  structures.  This  ensures  that  the  solution  to 
currently  limited  resources  within  individual  organizations  is  actually  achieved. 

Wider  Data  Availability 

This  document  describes  the  breadth  and  depth  of  personnel  data  collected  in  an  ongoing 
and  ad  hoc  manner,  although  a  comprehensive  census  of  all  such  projects  in  the  Air  Force  was 
beyond  the  scope  of  this  paper.  Moreover,  the  number  of  organizations  we  found  engaging 
in  such  research  and  the  numerous  studies  still  speaks  to  the  extensiveness  of  the  Air  Force’s 
efforts.  Much  of  the  data  collected  may  be  used  for  multiple  purposes.  However,  they  currently 
are  not,  for  a  variety  of  reasons.  A  key  component  of  change  efforts  would  be  to  smooth  the 
path  for  wider  data  availability.  The  oversight  and  coordination  components  described  above 
would  go  a  long  way  to  alleviating  the  lack  of  centralized  knowledge  of  the  existence  and 
nature  of  data- collection  efforts.  The  resource  component  would  enable  data,  once  known, 
to  be  shared,  by  funding  personnel  time  to  actually  manage  the  databases  and  navigate  the 
legal  and  privacy  requirements  for  these  data  linkages  to  occur.  However,  this  too  should  be 
an  explicit  mandate.  Currently,  no  one  organization’s  primary  mission  is  to  share  all  collected 
data  with  others,  and,  in  supporting  primary  missions,  the  priority  status  given  to  doing  the 
basic  work  of  sharing  data  resources  is  understandably  low.  A  good  solution  to  the  issues  of  the 
Air  Force’s  personnel  research  concerns  would  enable  data  to  be  shared  more  efficiently  with¬ 
out  upsetting  other  vital  priorities  and  missions.  Additional  personnel  to  organize  the  effort 
and  lighten  the  load  on  organizations  by  taking  on  such  nonessential  tasks  as  formatting  data 
sets  for  sharing  (including  alleviating  concerns  about  confidentiality  of  the  data  and  ensuring 
security)  would  facilitate  the  data-sharing  process. 

Increased  Visibility  to  the  Wider  Air  Force 

Last  but  not  least,  a  component  of  the  change  would  need  to  be  increased  visibility  to  the  wider 
Air  Force  for  the  organizational  structure  chosen.  Currently,  some  personnel  research  organi¬ 
zations  in  the  Air  Force  have  relatively  high  visibility,  while  others  do  not;  any  coordinating 
structure  would  need  sufficient  visibility  for  various  proponents  in  the  Air  Force  to  easily  find 
the  structure  and  recognize  it  as  an  Air  Force-wide  resource,  as  well  as  a  structure  serving  a 
clearinghouse  function.  Visibility  would  in  fact  be  key  to  enabling  the  organizational  structure 
to  serve  as  a  clearinghouse  because  it  is  probable  that  there  are  ongoing  projects  undertaken  by 
organizations  with  relatively  little  “official”  or  formal  connection  to  the  Air  Force  personnel 
research  system  that  are  engaging  in  relevant  efforts.  Although  a  personnel  research  organiza¬ 
tion  may  not  have  the  capacity  or  authority  to  conduct  the  research  requested  itself,  it  could 
advise  on  such  matters  and  direct  questions  to  appropriate  organizations.  This  visibility  com¬ 
ponent  may  also  imply  that  some  marketing  of  organizational  purpose  and  resources  would 
need  to  be  done  on  an  ongoing  basis  on  behalf  of  the  organizational  structure. 


Applied  Scenarios 

Ultimately,  the  Air  Force  leadership  must  determine  its  own  course  of  action  with  regard  to 
aligning  the  ongoing  studies,  analysis,  and  research  on  personnel  issues  to  better  support  its 
mission  and  vision.  Thus,  we  suggest  components  necessary  (oversight,  authority,  institutional 
knowledge,  quality  control,  expertise  and  other  resources,  data  availability,  and  visibility)  in 
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an  organizational  structure  for  that  structure  to  provide  the  solution  to  the  current  gaps  we 
perceive  in  the  Air  Force  personnel  research  system:  narrow  missions,  inconsistent  data  sharing 
and  coordination,  lack  of  expertise,  limited  resources,  reliance  on  contractors,  and  duplication 
of  effort.  The  Air  Force  itself  must  determine  its  own  proper  course  to  enable  change  to  he  pos¬ 
sible  and  tenable  for  all  stakeholders  involved. 

In  all  cases,  we  would  suggest  that  at  least  some  of  the  funding  for  the  new  structure  be 
from  MFP  6,  slated  for  science  and  technology  research  and  development.  This  would  enable 
the  new  organizational  structure  to  take  on  the  task  of  strategic  research  and  development  that 
has  been  lacking  in  the  personnel  system  as  a  whole  since  the  last  human  resources-centric 
component  of  the  Armstrong  Laboratory  was  dismantled  during  the  adjustment  under  AFRL. 
Historically  within  the  Armstrong  Lab  and  today  at  AFRL,  some  of  the  health-related  work, 
such  as  that  conducted  by  USAFSAM,  is  funded  through  non-research  and  development  (i.e., 
non-MFP  6)  funds.  Although  this  integration  with  the  research  and  development-funded  lab 
structures  was  considered  a  bit  unusual,  the  synergy  worked  in  the  past  (Duffner,  2000)  and, 
as  the  current  situation  at  AFRL  demonstrates,  could  work  again. 

Although  the  way  forward  must  be  determined  internally  by  the  Air  Force,  it  is  still  useful 
to  present  some  example  scenarios  of  possible  approaches  the  Air  Force  may  undertake  to  meet 
its  needs.  These  alternatives  may  serve  as  a  tool  to  engender  useful  discussion  among  key  stake¬ 
holders.  Thus,  we  suggest  three  potential  approaches:  We  will  call  these  the  baseline  plus  one 
scenario,  the  comprehensive  alignment  scenario,  and  the  hybrid  scenario.  In  developing  these  sce¬ 
narios,  we  attempted  to  address  the  range  of  possibilities,  including  the  minimal  intervention 
{baseline  plus  one  scenario,  so  called  because  it  adds  only  a  minimal  oversight  component — the 
plus  one — to  the  overall  organizational  structure  to  address  the  gaps  we  noted),  the  maximal 
intervention  {comprehensive  alignment  scenario,  so  called  because  it  involves  substantial  reorga¬ 
nization  to  align  personnel  research),  and  a  middle  option  {hybrid  scenario,  so  called  because 
it  is  a  hybrid  of  the  two  extremes).  Each  of  these  will  include  the  components  for  change  that 
address  the  identified  gaps.  Because  the  current  effort  focused  on  existing  personnel  research 
organizations  in  the  Air  Force,  and  because  these  functions  are  likely  to  continue  in  some 
fashion  in  the  immediate  future,  our  alternatives  focus  similarly  on  these  organizations  and 
functions.  Reorganization  and  organizational  change  never  take  place  in  a  vacuum,  and  cer¬ 
tain  factors  require  consideration  when  plotting  the  future  course.  These  include  the  interre¬ 
lated  issues  of  cost,  convenience  of  implementation,  potential  for  effectiveness,  continuation 
of  current  good  efforts,  and  quality  of  current  and  future  efforts.  As  in  project  management’s 
concept  of  the  triple  constraint  (projects  can  be  economical,  high  quality,  or  fast,  but  one  can 
achieve  only  two  out  of  three;  see,  e.g.,  Kernzer,  2009),  trade-offs  must  be  made  in  the  decision 
of  what  organizational  structure  to  utilize.  For  example,  the  cheapest  option  is  not  always  the 
best  option  in  terms  of  actually  enabling  the  potential  for  effectiveness.  Thus  we  use  the  three 
constraints  as  a  lens  through  which  to  view  the  potential  approaches  to  better  aligning  the 
work  being  done  by  organizations  within  the  Air  Force. 

Baseline  Plus  One  Scenario 

First,  these  organizations  may  continue  as  at  present,  in  their  existing  hierarchies,  with  their  cur¬ 
rent  missions  and  functions.  The  oversight  organizational  structure  could  be  composed  of  rela¬ 
tively  few  key  personnel  to  provide  for  the  various  key  components  that  resolve  the  concerns 
we  noted.  These  personnel  would  have  to  have  the  requisite  expertise  to  serve  as  personnel 
research  SMEs  capable  of  identifying  gaps  caused  by  lack  of  coordination  and  narrow  mis- 
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sions,  to  provide  expertise  when  needed  on  personnel  research  issues,  and  to  enable  the  quality 
control  component.  Moreover,  one  or  more  people  within  the  structure  would  need  to  embody 
institutional  knowledge  of  the  Air  Force  and  existing  personnel  research  mechanisms  to  best 
leverage  existing  research,  studies,  and  resources  more  generally.  Some  staff  would  be  necessary 
to  facilitate  data  coordination  and  sharing  among  various  organizations  to  provide  for  the  com¬ 
ponent  of  wider  data  availability.  Finally,  there  would  need  to  be  a  sufficient  number  of  per¬ 
sonnel,  funded  appropriately,  to  enable  a  more  strategic  focus,  manage  the  oversight  function 
itself,  facilitate  coordination  and  data  sharing,  and  promote  visibility.  So,  although  this  option 
considers  a  relatively  small  number  of  personnel,  examination  of  the  demands  inherent  in  the 
task  dictates  that  even  a  small  oversight  organizational  structure  would  require  several  person¬ 
nel  with  various  functions  and  capacities  to  enable  success.  Authority  for  such  a  structure  could 
be  made  explicit  in  official  doctrine  and  enforced  via  contingencies  that  allow  the  organization 
some  control  over  funding  or  employment  decisionmaking  for  the  other  organizations  involved 
in  the  personnel  research  system. 

In  the  spirit  of  openly  confronting  potential  barriers  to  change,  effectiveness  and  strategic 
alignment  of  the  personnel  research  undertaken  are  less  likely  to  be  achieved  in  this  course,  in 
part  because  existing  stovepipes  are  maintained  and  underestimating  the  resources  required  to 
overcome  them  is  a  temptation  that  would  be  difficult  to  counter.  Insufficient  resources  would 
not  just  ensure  failure  but,  in  effect,  add  a  useless  layer  of  bureaucracy  to  the  already-complex 
process  of  workforce  planning  and  management,  rather  than  adding  a  true  asset  and  strategic 
partner.  The  costs  of  potential  continuing  duplication  of  effort  and  lack  of  quality  control  over¬ 
all  are  hard  to  estimate  but,  as  noted,  are  likely  to  be  expensive;  add  to  that  the  cost  of  funding 
another  layer  of  bureaucracy  rather  than  an  organization  empowered  to  meet  its  mission  of 
strategic  alignment  of  personnel  research.  However,  the  maintenance  of  existing  organizational 
structures  does  make  this  exemplar  course  of  action  easier  to  implement.  This  path  optimizes 
cost  and  convenience  because  it  is  relatively  inexpensive,  requires  no  inconvenient  changes,  and 
would  enable  the  undisturbed  continuation  of  high-quality  ongoing  research.  In  terms  of  the 
triple  constraints,  this  course  maximizes  speed  and  economics  rather  than  quality. 

Comprehensive  Alignment  Scenario 

A  second  alternative  is  that  the  various  personnel  research  organizations  and  functions  discussed  in 
this  document  could  be  reorganized  under  a  single  oversight  organization.  This  has  the  advantage 
of  placing  all  the  stakeholders  together  organizationally  and  emphasizing  the  commonalities 
rather  than  the  differences,  which  would  ideally  facilitate  communication  and  coordination  of 
research  itself,  as  well  as  enabling  wider  data  availability.  Moreover,  this  has  the  advantage  of 
creating  a  central  organization  with  an  explicit  responsibility  for  aligning  the  personnel  system 
with  the  research  that  should  support  decisionmaking.  This  enables  the  oversight  component 
and  creates  an  organization  with  sufficient  breadth  of  mission  to  integrate  incoming  informa¬ 
tion  from  current  structures  with  relatively  narrow  mission  sets.  Institution  of  an  appropriate 
performance  management  system  within  the  organization  would  provide  additional  impetus 
to  official  oversight  authority.  Institutional  knowledge  would  be  retained  within  the  organiza¬ 
tion,  though  an  investment  in  additional  billets  would  still  need  to  be  made  for  personnel  to 
serve  as  various  strategic  SMEs  and  provide  for  the  quality  control  component  and  to  provide 
relevant  personnel  research  expertise.  Additional  billets  would  also  still  be  needed  to  facilitate 
actual  data  sharing  without  overburdening  existing  personnel,  although  it  is  possible  that,  as 
greater  data-collection  efficiency  is  achieved,  fewer  people  will  be  needed  to  manage  and  coor- 
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dinate  existing  data.  Because  all  personnel  research  would  be  centered  here,  it  is  likely  that  the 
visibility  component  would  be  achieved  as  a  matter  of  course;  however,  care  should  be  taken 
that  the  organization  is  sufficiently  high  in  the  organizational  structure  of  the  Air  Force,  or 
sufficiently  well-publicized,  that  it  may  serve  as  a  beacon. 

Complications  attend  this  exemplar  scenario  as  well.  The  old  AFHRL  was  criticized  for  a 
lack  of  responsiveness  to  managers  on  the  ground  and  in  the  field  and  to  policymakers  them¬ 
selves.  Some  of  our  interlocutors  suggested  that  AFHRL  was  an  ivory  tower,  disconnected 
from  the  broader  needs  of  the  Air  Force,  and  that  perhaps  this  disconnect  is  what  ultimately 
led  to  its  demise.  It  would  be  vitally  important  for  a  central  personnel  research  organization 
to  remain  integrated  with  the  mission  of  the  Air  Force  as  a  whole,  inclusive  of  emergent  con¬ 
cerns,  to  forestall  this  issue.  Careful  consideration  of  the  appropriate  contingencies  to  facilitate 
responsiveness  to  the  needs  of  policymakers  while  retaining  sufficient  rewards  for  engaging  in 
a  strategic  approach  would  be  necessary.  This  approach  might  also  require  increased  communi¬ 
cation  with  operational  stakeholders  to  clearly  draw  the  parallels  between  the  research  and  its 
applications.  A  research  organization  may  suffer  here:  The  prototypical  researcher  is  not  known 
for  “people”  skills,  and  this  is  true  even  for  scientists  in  person-focused  disciplines,  such  as 
psychology.  A  failure  of  communication  and  responsiveness  on  the  part  of  this  organizational 
structure  raises  the  specter  of  an  Air  Force  determination  that  an  organization  that  supports 
a  personnel  system  research  application  for  strategic  alignment  is  not  truly  meeting  the  needs 
of  the  Air  Force  and  is  instead  a  “resource  sink”  that  does  not  facilitate  the  strategic  goals  of 
the  service.  Because  personnel  research  is  necessary  to  ensure  that  decisionmakers  have  data  on 
which  to  rest  their  recommendations,  this  possibility  is  a  definite  concern. 

This  course  offers  some  definite  positives.  Good  research  is  likely  to  be  continued,  and 
future  research  is  likely  to  be  given  the  coordinated  time,  attention,  strategic  focus,  and  exper¬ 
tise  needed.  Putting  everything  together  under  one  roof  maximizes  the  potential  for  strate¬ 
gic  alignment  and  effective  coordination  although,  as  exemplified  in  the  macro  literature  on 
downsizing,  there  would  likely  be  an  adjustment  period  of  at  least  a  year  or  two  before  gains 
would  be  perceptible  (De  Meuse,  Marks,  and  Dai,  2011).  Moreover,  funding  streams  would 
be  no  simple  matter  because  part  of  the  strategic  alignment  and  research  and  development 
mandate  would  necessitate  MFP  6  funding  in  tandem  with  other  types  of  funding,  such  as 
MFP  8.  Although  such  funding  combinations  have  worked  in  the  past  and  continue  to  work  in 
the  present  (e.g.,  aerospace  medicine  and  health  sciences  units,  such  as  USAFSAM,  with  other 
research  at  the  Armstrong  Laboratory,  as  well  as  AFRL;  Duffner,  2000),  they  are  far  from  the 
customary  way  of  doing  business  in  the  Air  Force,  and  due  care  would  need  to  be  exercised  to 
make  the  combination  tenable. 

Additional  trade-offs  inhere  in  this  example  scenario.  Given  the  current  entrenched  stove¬ 
pipes  and  issues  of  moving  organizations,  as  well  as  the  required  resources  in  terms  of  expertise, 
it  would  be  quite  expensive  in  terms  of  time  and  money  to  implement  this  course  and  allow 
sufficient  time  for  improved  performance  to  manifest.  Moreover,  to  avoid  some  of  AFHRL’s 
past  problems,  further  resources  would  likely  be  required  for  a  marketing  and  communication 
element.  This  course  maximizes  quality  and  would  likely  not  be  either  fast  or  economical. 

Hybrid  Scenario 

A  third  example  structure  scenario,  a  hybrid  approach,  could  also  be  taken.  For  example,  as 
shown  in  Figure  6.1,  a  new  Personnel  Research  Directorate  with  a  division  focused  more  nar¬ 
rowly  on  JA-type  data  collection  could  house  OAD  and  AFMA  MAS,  and  a  second  divi- 
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sion  could  incorporate  the  other,  more  disparate  elements  of  AFMA,  AAD,  and  the  DSYD, 
DSYA,  and  SRA  currently  at  AFPC.  This  would  put  all  the  organizations  regularly  conducting 
operational  personnel  research  together.  As  shown  here,  the  strategic  mission  could  he  worked 
into  the  personnel  overseeing  the  new  directorate,  with  research  and  development  funding 
utilized  as  appropriate.  This  would  resolve  concerns  regarding  individual  narrow  missions. 
Additional  personnel  within  the  directorate  management  could  fulhll  the  needed  requirements 
for  personnel  research  suhject-matter  expertise  and  institutional  knowledge,  which  would  still 
enable  appropriate  leveraging  of  resources  to  fill  remaining  identified  gaps  through  contracting 
for  external  skill  sets  or  achieving  synergies  within  the  new  organizational  structure.  Benefits 
include  the  advantages  that  accrue  from  collocating  (organizationally,  if  not  geographically) 
organizations  that  have  personnel  research  as  a  shared  activity.  Oversight  is  easier,  and  plac¬ 
ing  them  in  the  same  organizational  structure  enables  creation  of  a  performance  management 
system  designed  to  reward  cooperation  and  coordination  of  data  and  effort  and  to  enable  the 
oversight  personnel  to  execute  both  their  authority  and  mandate  for  quality  control.  Because 
many  of  these  organizations  are  located  in  San  Antonio,  collocation  organizationally  need  not 
disrupt  the  personal  lives  of  the  employees  and  result  in  a  shedding  of  experienced  workforce 
and  a  loss  of  institutional  knowledge,  as  has  been  a  concern  for  base  realignment  and  closure 
(e.g.,  Masi  et  ah,  2009).  AFPC  is  the  FOA  with  the  responsibility  for  maintaining  the  Air 
Force’s  personnel  system  and  has  visibility  in  that  regard,  which  would  partially  meet  the  vis¬ 
ibility  needs  of  the  new  organizational  structure.  Finally,  these  organizations  collectively  also 
have  a  history  of  responsiveness  to  the  needs  of  their  superior  organizations,  which  should 
transfer  to  some  degree. 

The  needed  mandate  for  a  strategic  approach  does  add  some  complications  to  this  solu¬ 
tion  because  the  research  and  development  functionality  in  the  Air  Force  is  situated  under  the 
acquisition  community  rather  than  with  the  personnel  community  and  is  located  at  AFRL 
in  Dayton,  Ohio,  rather  than  with  the  personnel  organizations  in  San  Antonio.  Moreover, 
integration  of  the  ongoing  training  research  and  development  work  being  conducted  at  AFRL 
today  within  the  711  HPW  would  be  necessary  to  truly  bring  all  of  the  personnel  research 
endeavors  into  alignment.  Although  aligning  OAD  more  closely  with  the  other  personnel 
organizations  located  at  San  Antonio  would  further  integrate  the  application  of  OA  to  the  per¬ 
sonnel  research  process  within  the  Air  Force,  this  alone  would  not  integrate  the  full  breadth  of 
training  personnel  research  being  done. 

It  should  be  explicitly  noted  that  merely  collocating  these  organizations  would  not  negate 
the  need  to  hire  additional  personnel  to  coordinate  and  communicate  among  the  organizations 
and  develop  and  execute  a  strategic  personnel  research  plan;  in  some  senses,  the  needs  would 
be  more  acute  because  the  existing  organizations,  maintained  in  the  overall  structure,  have 
historically  been  stovepiped  in  their  roles  and  objectives.  Maintaining  existing  structures  with 
such  a  history  would  transfer  organizational  climate  and  culture  in  its  entirety.  Although  this 
could  serve  as  a  benefit  in  the  sense  of  transferring  responsiveness  to  the  Air  Force  mission,  it 
could  serve  as  a  drawback  in  the  sense  that  the  inclination  to  remain  stovepiped  would  transfer 
as  well.  As  with  a  fully  integrated  new  organization,  communication  with  the  wider  Air  Force 
would  still  be  necessary,  especially  to  highlight  the  changes  and  to  advertise  the  effectiveness 
engendered  by  any  synergy,  as  well  as  to  enable  the  organization  to  serve  its  clearinghouse 
function  via  its  sheer  visibility.  The  strategic  research  element  should  be  located  with  the  rear¬ 
ranged  organizations  in  San  Antonio  to  fully  optimize  the  geographic  advantage.  That  said, 
an  outpost  in  Dayton  with  a  few  personnel,  or  explicitly  identified  personnel  in  both  the  new 
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Figure  6.1 

Example  of  an  Air  Force  Personnel  Center-Based  Structure  Tying  Together  Key  Personnel  Research 
Components 


divisions 

RAND  TR1276-6.1 

directorate  and  at  AFRL  to  serve  as  liaisons,  would  unite  the  broad  strategic  research  element 
coordinating  the  other  personnel  research  endeavors  with  the  ongoing  training  research  and 
development  and  facilitate  science  and  technology  synergy.  Whatever  the  course  chosen  in  this 
particular  regard,  a  linkage  to  the  711  HPW  is  advisable  and  is  shown  in  Figure  6.1  with  a 
dashed  line. 

In  terms  of  cost,  this  option  is  a  middle  road  because  additional  resources  would  still 
be  required  for  coordination,  data  sharing,  and  expertise  and  to  fill  data-collection  gaps.  It  is 
somewhat  convenient  to  implement  in  the  sense  of  existing  geographic  collocation,  but  over¬ 
coming  stovepipes  would  still  require  time  and  effort.  However,  this  does  add  the  definite 
benefits  of  needed  expertise  to  monitor  quality  and  resources  to  help  align  personnel  research 
more  strategically,  and  integrating  those  resources  within  the  organizational  element  should 
avert  issues  of  those  resources  being  simply  another  bureaucratic  layer.  Current  good  efforts  are 
likely  to  be  maintained.  This  course  does  not  explicitly  optimize — or  sacrifice — cost,  speed, 
or  quality. 

Although  none  of  the  three  constraints  is  maximized,  this  course  would  move  the  Air 
Force  closer  to  an  optimization  of  the  personnel  research  system  and  has  compelling  benefits 
in  terms  of  convenience.  In  fact,  should  the  Air  Force  at  some  point  in  the  future  determine 
that  complete  strategic  alignment  of  the  personnel  research  system  is  a  goal  it  wishes  to  pursue, 
this  course  would  serve  to  make  that  final  move  less  painful  in  terms  of  cost  and  speed  of 
implementation . 

As  of  this  writing,  the  Air  Force  is  already  moving  in  this  direction,  merging  AFPC, 
AFMA,  and  AFSVA,  the  Air  Force  FOA  whose  primary  mission  is  a  service  and  support  func¬ 
tion,  providing  many  benefit  programs  and  services,  such  as  gyms  and  exchanges.  However, 
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the  extent  to  which  the  coordination  of  personnel  research  is  a  goal  or  even  a  consideration  in 
this  reorganization  is  unknown.  Certainly,  incorporation  of  the  long-term  strategic  alignment 
of  personnel  research  is  not,  to  our  knowledge,  currently  a  part  of  the  design  and  on  no  one’s 
agenda. 


Final  Thoughts 

Overall,  we  found  that  the  alignment  within  the  personnel  research  system  that  should  he  sup¬ 
porting  the  Air  Force’s  personnel  policy  decisionmaking  is  currently  imperfect.  We  identihed 
several  concerns:  narrow  missions,  inconsistent  data  sharing  and  coordination,  lack  of  exper¬ 
tise,  limited  resources,  reliance  on  contractors,  and  duplication  of  effort.  Our  solution  entails 
a  set  of  essential  components  for  change  (oversight,  authority,  institutional  knowledge,  quality 
control,  expertise  and  other  resources,  data  availability,  and  visibility),  and  we  explicated  these 
through  application  to  three  potential  scenarios  the  Air  Force  may  undertake,  ranging  from  a 
baseline  plus  one  through  comprehensive  alignment.  These  vary  in  terms  of  the  priority  placed 
on  optimizing  cost,  speed,  or  quality,  as  seen  in  Table  6.1.  In  the  table,  black  indicates  the 
approach  with  the  greatest  amount  of  optimization  of  a  given  constraint  factor;  gray,  moderate 
optimization;  and  white,  nonoptimization. 

Organizational  change  is  a  nontrivial  matter.  Hedge  and  Pulakos  (2002)  cite  research 
that  suggests  that  approximately  70  percent  of  change  initiatives,  such  as  Total  Quality  Man¬ 
agement,  fail.  Beer  and  Eisenstat  (1996)  examined  the  organizational  change  literature  and 
summed  it  up  with  three  recommendations  for  successful  implementation: 

•  Organizational  change  should  be  systemic — that  is,  any  organizational  change  should 
be  framed  in  terms  of  the  organizational  system  in  which  it  occurs  and  properly  aligned 
with  the  organization.^ 

•  Barriers  to  the  change  process  should  be  openly  addressed  in  the  process.  That  is,  there  are 
many  reasons  for  resistance  to  organizational  change,  including  comfort  with  the  current 
system  and  the  threatened  prerogatives  of  various  elements  within  the  system.  Beer  and 
Eisenstat  note  that  it  is  impossible  to  develop  a  realistic  plan  for  implementing  change 
without  a  similarly  realistic  approach  to  this  resistance. 

•  Stakeholder  buy-in  and  mutual  partnerships  are  key  because  these  stakeholders  are  ulti¬ 
mately  responsible  for  how  change  initiatives  are  implemented. 

Table  6.1 

Notional  Optimization  of  Triple  Constraint  Factors 

Baseline  Plus  One  Comprehensive 

Factor  Scenario  Alignment  Scenario  Flybrid  Scenario 

Economics 

Quality 

Speed 


^  Note  that  this  recommendation  echoes  the  recommendations  provided  by  GAO  and  OPM  about  human  capital  man¬ 
agement,  as  well  as  the  literature  presented  in  Chapter  Two. 


64  Air  Force  Personnel  Research:  Recommendations  for  Improved  Alignment 


Ultimately,  bringing  the  personnel  research  system  of  the  Air  Force  into  alignment  is 
an  exercise  in  organizational  change.  Even  the  most  economical  and  quick  change  the  Air 
Force  might  make  would  require  sustained  commitment  from  leadership  at  the  highest  levels. 
However,  given  the  vast  nature  of  the  personnel  system  itself,  and  the  costs  that  inhere  in 
the  system,  aligning  the  personnel  research  system  to  better  support  strategic  decisionmaking 
offers  the  potential  for  large  dividends,  particularly  in  a  time  of  resource  constraints. 
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